71 results back to index
barriers to entry, business intelligence, business process, call centre, cloud computing, commoditize, en.wikipedia.org, Just-in-time delivery, knowledge worker, Richard Stallman, software as a service, statistical model, supply-chain management, the market place
In addition, there are several more companies that participated in case studies by sharing their experiences and outlook on open source deployments and their use of business intelligence without their input it would not have been possible to provide insight into the real world applications of open source business intelligence. I would also like to thank Cindi Howson, Howard Dresner, David Loshin, Shawn Rogers, and Wayne Eckerson for sharing their experiences and advice unrestrictedly. And lastly, I would like to thank Gail Wise, for providing advice, support and lending an ear throughout the writing and reviewing process. xv This page intentionally left blank SECTION Introduction and market overview 1 This page intentionally left blank CHAPTER Introducing BI 1 Why is there a need for this book? Many books exist that identify how to get the most out of analytics or how to develop an open source business intelligence (OSBI) solution based on specific development or solution requirements.
This choice means looking at whether an organization will implement a full BI solution, consider a best-of-breed alternative, look at SaaS, or customize an analytics environment through the use of OS. Once an organization determines this, they can narrow their search to the solutions that meet these needs. Although the other delivery methods are outside the scope of this book, your company may still want to consider a mix of solutions. Therefore, it makes sense to look at how business intelligence and OS intersect. The components of business intelligence Before we expand to look at the OS market, let’s look a little deeper at how business intelligence pieces fit together by taking a closer look at Figure 1-1 above. The data warehouse represents where all of the information for later use is stored. Figure 1-2 shows this on a high level. The data marts so valued by both Inmon and Kimball are what make up the whole of the data warehouse. While information can be stored within the data warehouse even if it isn’t being used within a specific data mart, the way in which data is captured, stored, and processed is outside the scope of this book as the aim here is to give you a very broad understanding of BI and how each component fits within a bigger whole.
Using Open Source Platforms for Business Intelligence This page intentionally left blank Using Open Source Platforms for Business Intelligence Avoid Pitfalls and Maximize ROI Lyndsay Wise AMSTERDAM • BOSTON • HEIDELBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO Morgan Kaufmann is an imprint of Elsevier Acquiring Editor: Andrea Dierna Development Editor: Robyn Day Project Manager: Paul Gottehrer Designer: Alisa Andreola Morgan Kaufmann is an imprint of Elsevier 225 Wyman Street, Waltham, MA 02451, USA Copyright r 2012 Elsevier Inc. All rights reserved No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher.
Big Data Analytics: Turning Big Data Into Big Money by Frank J. Ohlhorst
algorithmic trading, bioinformatics, business intelligence, business process, call centre, cloud computing, create, read, update, delete, data acquisition, DevOps, fault tolerance, linked data, natural language processing, Network effects, pattern recognition, performance metric, personalized medicine, RFID, sentiment analysis, six sigma, smart meter, statistical model, supply-chain management, Watson beat the top human players on Jeopardy!, web application
Titles in the Wiley and SAS Business Series include: Activity-Based Management for Financial Institutions: Driving Bottom-Line Results by Brent Bahnub Advanced Business Analytics: Creating Business Value from Your Data by Jean Paul Isson and Jesse Harriott Branded! How Retailers Engage Consumers with Social Media and Mobility by Bernie Brennan and Lori Schafer Business Analytics for Customer Intelligence by Gert Laursen Business Analytics for Managers: Taking Business Intelligence beyond Reporting by Gert Laursen and Jesper Thorlund The Business Forecasting Deal: Exposing Bad Practices and Providing Practical Solutions by Michael Gilliland Business Intelligence Success Factors: Tools for Aligning Your Business in the Global Economy by Olivia Parr Rud CIO Best Practices: Enabling Strategic Value with Information Technology, Second Edition by Joe Stenzel Connecting Organizational Silos: Taking Knowledge Flow Management to the Next Level with Social Media by Frank Leistner Credit Risk Assessment: The New Lending System for Borrowers, Lenders, and Investors by Clark Abrahams and Mingyuan Zhang Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring by Naeem Siddiqi The Data Asset: How Smart Companies Govern Their Data for Business Success by Tony Fisher Demand-Driven Forecasting: A Structured Approach to Forecasting by Charles Chase Executive’s Guide to Solvency II by David Buckham, Jason Wahl, and Stuart Rose The Executive’s Guide to Enterprise Social Media Strategy: How Social Networks Are Radically Transforming Your Business by David Thomas and Mike Barlow Fair Lending Compliance: Intelligence and Implications for Credit Risk Management by Clark R.
See Business analytics (BA) BackType Backup systems Batch processing Behavioral analytics Benefits analysis Best practices anomalies expediency-accuracy tradeoff high-value opportunities focus in-memory processing project management processes project prerequisites thinking big worst practice avoidance BI. See Business intelligence (BI) Big Data and Big Data analytics analysis categories application platforms best practices business case development challenges classifications components defined evolution of examples of 4Vs of goal setting introduction investment in path to phases of potential of privacy issues processing role of security (See Security) sources of storage team development technologies (See Technologies) value of visualizations Big Science BigSheets Bigtable Bioinformatics Biomedical industry Blekko Business analytics (BA) Business case best practices data collection and storage options elements of introduction Business intelligence (BI) as Big Data analytics foundation Big Data analytics team incorporation Big Data impact defined extract, transform, and load (ETL) information technology and in-memory processing limitations of marketing campaigns risk analysis storage capacity issues unstructured data visualizations Business leads Business logic Business objectives Business rules C Capacity of storage systems Cassandra Census data CERN Citi Classification of data Cleaning Click-stream data Cloud computing Cloudera Combs, Nick Commodity hardware Common Crawl Corpus Communication Competition Compliance Computer security officers (CSOs) Consulting firms Core capabilities, data analytics team Costs Counterintelligence mind-set CRUD (create, retrieve, update, delete) applications Cryptographic keys Culture, corporate Customer needs Cutting, Doug D Data defined growth in volume of value of See also Big Data and Big Data analytics Data analysis categories challenges complexity of as critical skill for team members data accuracy evolution of importance of process technologies Database design Data classification Data discovery Data extraction Data integration technologies value creation Data interpretation Data manipulation Data migration Data mining components as critical skill for team members defined examples methods technologies Data modeling Data protection.
For more information about Wiley products, visit www.wiley.com. Library of Congress Cataloging-in-Publication Data: Ohlhorst, Frank, 1964– Big data analytics : turning big data into big money / Frank Ohlhorst. p. cm. — (Wiley & SAS business series) Includes index. ISBN 978-1-118-14759-7 (cloth) — ISBN 978-1-118-22582-0 (ePDF) — ISBN 978-1-118-26380-8 (Mobi) — ISBN 978-1-118-23904-9 (ePub) 1. Business intelligence. 2. Data mining. I. Title. HD38.7.O36 2013 658.4'72—dc23 2012030191 Preface What are data? This seems like a simple enough question; however, depending on the interpretation, the definition of data can be anything from “something recorded” to “everything under the sun.” Data can be summed up as everything that is experienced, whether it is a machine recording information from sensors, an individual taking pictures, or a cosmic event recorded by a scientist.
The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling by Ralph Kimball, Margy Ross
active measures, Albert Einstein, business intelligence, business process, call centre, cloud computing, data acquisition, discrete time, inventory management, iterative process, job automation, knowledge worker, performance metric, platform as a service, side project, supply-chain management, zero-sum game
We’ll begin with a primer on DW/BI and dimensional modeling in Chapter 1 to ensure that everyone is on the same page regarding key terminology and architectural concepts. 1 Data Warehousing, Business Intelligence, and Dimensional Modeling Primer T his first chapter lays the groundwork for the following chapters. We begin by considering data warehousing and business intelligence (DW/BI) systems from a high-level perspective. You may be disappointed to learn that we don’t start with technology and tools—first and foremost, the DW/BI system must consider the needs of the business. With the business needs firmly in hand, we work backwards through the logical and then physical designs, along with decisions about technology and tools. We drive stakes in the ground regarding the goals of data warehousing and business intelligence in this chapter, while observing the uncanny similarities between the responsibilities of a DW/BI manager and those of a publisher.
Business users are underwhelmed by the usability and performance provided by these pseudo data warehouses; these imposters do a disservice to DW/ BI because they don’t acknowledge their users have drastically different needs than operational system users. Data Warehousing, Business Intelligence, and Dimensional Modeling Primer Goals of Data Warehousing and Business Intelligence Before we delve into the details of dimensional modeling, it is helpful to focus on the fundamental goals of data warehousing and business intelligence. The goals can be readily developed by walking through the halls of any organization and listening to business management. These recurring themes have existed for more than three decades: ■ ■ ■ ■ ■ ■ “We collect tons of data, but we can’t access it.” “We need to slice and dice the data every which way.”
NOTE Data in the queryable presentation area of the DW/BI system must be dimensional, atomic (complemented by performance-enhancing aggregates), business process-centric, and adhere to the enterprise data warehouse bus architecture. The data must not be structured according to individual departments’ interpretation of the data. Business Intelligence Applications The ﬁnal major component of the Kimball DW/BI architecture is the business intelligence (BI) application. The term BI application loosely refers to the range of capabilities provided to business users to leverage the presentation area for analytic decision making. Data Warehousing, Business Intelligence, and Dimensional Modeling Primer 23 By definition, all BI applications query the data in the DW/BI presentation area. Querying, obviously, is the whole point of using data for improved decision making. A BI application can be as simple as an ad hoc query tool or as complex as a sophisticated data mining or modeling application.
Big Data at Work: Dispelling the Myths, Uncovering the Opportunities by Thomas H. Davenport
Automated Insights, autonomous vehicles, bioinformatics, business intelligence, business process, call centre, chief data officer, cloud computing, commoditize, data acquisition, Edward Snowden, Erik Brynjolfsson, intermodal, Internet of things, Jeff Bezos, knowledge worker, lifelogging, Mark Zuckerberg, move fast and break things, move fast and break things, Narrative Science, natural language processing, Netflix Prize, New Journalism, recommendation engine, RFID, self-driving car, sentiment analysis, Silicon Valley, smart grid, smart meter, social graph, sorting algorithm, statistical model, Tesla Model S, text mining
The general activity of making sense of data has been called decision support, executive support, online analytical processing, business intelligence, analytics, and now big data (see table 1-3).4 There are certainly some new elements in each generation of terminology, but I’m not sure that things have evolved enough to be worthy of six generations. Ta b l e 1 - 3 Terminology for using and analyzing data Term Time frame Specific meaning Decision support 1970–1985 Use of data analysis to support decision making Executive support 1980–1990 Focus on data analysis for decisions by senior executives Online analytical processing (OLAP) 1990–2000 Software for analyzing multidimensional data tables Business intelligence 1989–2005 Tools to support datadriven decisions, with emphasis on reporting Analytics 2005–2010 Focus on statistical and mathematical analysis for decisions Big data 2010–present Focus on very large, unstructured, fast-moving data Chapter_01.indd 10 03/12/13 3:24 AM Why Big Data Is Important to You and Your Organization 11 What makes big data somewhat worthy of a new term are the new and more voluminous forms of data that it involves—2.5 quintillion (that’s 2.5 followed by eighteen zeros) bytes per day generated around the world, by one estimate.5 Less structured data types, as I’ve suggested, are even more worthy of a new term and approach.
If there are intermediaries between data scientists and decision makers, the decision makers may not understand all the important data and analytics issues involved in a key decision. There is some evidence that these skills are important. A Gartner study found that “between 70 percent and 80 percent of c orporate business intelligence projects fail” owing to “a c ombination of poor communications between IT and the business, the failure to ask the right questions or to think about the real needs of the business.”3 And granted, business intelligence projects usually involve small rather than big data. However, while the specific percentage of projects that fail is questionable, there is no doubt that lack of communication in small and big data projects causes big problems. I won’t go into great detail on this topic, because I’ve coauthored a book called Keeping Up with the Quants that deals with it extensively.4 I don’t think there are major differences between the communication and trusted adviser skills of small data quants and data scientists; these skills are highly necessary in both jobs.
What Most Large Companies Do Today The classic analytics environment at most big companies includes the operational systems that serve as the sources for data; a data warehouse or collection of federated data marts that house and—ideally— integrate the data for a range of analysis functions; and a set of business intelligence and analytics tools that enable decisions from the use of ad hoc queries, dashboards, and data mining. Figure 5-4 illustrates the typical big company data warehouse ecosystem. Indeed, big companies have invested tens of millions of dollars in hardware platforms, databases, ETL (extract, transform, and load) software, BI (business intelligence) dashboards, advanced analytics tools, maintenance contracts, upgrades, middleware, and storage systems that comprise robust, enterprise-class data warehouse environments. In the best cases, these environments have helped companies understand their customer purchase and behavior patterns across channels and relationships, streamline sales processes, optimize Chapter_05.indd 128 03/12/13 1:04 PM Technology for Big Data 129 Figure 5-4 A typical data warehouse environment ERP Reporting CRM OLAP Legacy Third-party apps Data warehouse Ad hoc Modeling Source: SAS Best Practices.
23andMe, Affordable Care Act / Obamacare, Albert Einstein, big data - Walmart - Pop Tarts, bioinformatics, business intelligence, call centre, cloud computing, computer age, conceptual framework, Credit Default Swap, crowdsourcing, Daniel Kahneman / Amos Tversky, Danny Hillis, data is the new oil, David Brooks, East Village, Edward Snowden, Emanuel Derman, Erik Brynjolfsson, everywhere but in the productivity statistics, Frederick Winslow Taylor, Google Glasses, impulse control, income inequality, indoor plumbing, industrial robot, informal economy, Internet of things, invention of writing, John Markoff, John von Neumann, lifelogging, Mark Zuckerberg, market bubble, meta analysis, meta-analysis, money market fund, natural language processing, obamacare, pattern recognition, payday loans, personalized medicine, precision agriculture, pre–internet, Productivity paradox, RAND corporation, rising living standards, Robert Gordon, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, six sigma, skunkworks, speech recognition, statistical model, Steve Jobs, Steven Levy, The Design of Experiments, the scientific method, Thomas Kuhn: the structure of scientific revolutions, unbanked and underbanked, underbanked, Von Neumann architecture, Watson beat the top human players on Jeopardy!
Big data is such a market, and IBM recognized that years before the term became popular. The notion that computers should let you do something smart with data has been around since the dawn of computing. More recently, software programs for using data to make better-informed decisions were given the label “business intelligence.” It is a predecessor term to big data, and one still in use. Business intelligence tends to focus on collection, reporting, and basic analysis but not on the predictive or experimental features of data science. The concept dates back to 1958, presented in a paper titled, “A Business Intelligence System,” by Hans Peter Luhn, a computer scientist at IBM. Such a system, according to Luhn, was needed to cope with the postwar data boom in business, government, and the sciences. “Information,” he wrote, “is now being generated and utilized at an ever-increasing rate because of the accelerated pace and scope of human activities.”
The purpose, he wrote, was to digest and present information to guide decision making. “Special equipment,” he conceded, would be required. Luhn closed with a prescient bit of qualified optimism. “Perhaps the techniques which ultimately find greatest use will bear little resemblance to those now envisioned,” he wrote, “but some form of automation will ultimately provide an effective answer to business intelligence problems.” IBM’s headquarters is so nestled in woods and a rocky ravine that you don’t see it until well after you have entered the corporate grounds. The lazy Z–shaped building looks deceptively small from the outside. It is a striking contrast to the headquarters it replaced, a suburban corporate big house on a hill, opened in 1963. The new headquarters, far smaller, opened in 1997, makes a different statement, tucked into nature rather than rising above it.
It owns eight and manages them all, mostly in New York, with one each in Chicago, Miami, and Washington, DC; its brands include the Affinia hotels, and one-of-a-kind properties like The Surrey and The Benjamin. But the family-owned Denihan is an example of a conventional company that has made real progress toward using technology to make more of its decisions aided by data. And the hotel company has been at it seriously for more than a decade. Indeed, Denihan was cited as one of a few examples in a 2009 book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, by Howard Dresner, a respected business and technology consultant. The Denihan experience combines leadership at the top, focused goals, and close cooperation between a small data team and the people managing hotels day in and day out. The investment wasn’t huge, and the progress has come in measured steps. The current data program at Denihan had its origins in the late 1990s.
Business Metadata: Capturing Enterprise Knowledge by William H. Inmon, Bonnie K. O'Neil, Lowell Fryman
affirmative action, bioinformatics, business intelligence, business process, call centre, carbon-based life, continuous integration, corporate governance, create, read, update, delete, database schema, en.wikipedia.org, informal economy, knowledge economy, knowledge worker, semantic web, The Wisdom of Crowds, web application
Some DBMS vendors don’t supply an interface but instead supply utilities and application programming interfaces (APIs) so that you can write your own tools to extract metadata. Cautions that were expressed earlier in the section concerning ERP vendors also apply here: notably, when the vendor changes the system catalog structure, you must modify your tools accordingly. 7.2.6 Business Intelligence Tools Another good source for metadata is the Business Intelligence Tools (BI) environment—the user view of data served up by the data warehouse and data marts. The BI environment includes On-Line Analytical Processing (OLAP) tools. Such products produce cubes or pivot tables that enable drill-down and rollup analysis. Usually, OLAP multidimensional technology is found in the data mart environment. OLAP multidimensional tools usually keep a generous supply of metadata about their content, for example, information about how data has been defined to drill down and roll up.
Appendix Metadata System of Record Example Table 1 Metadata object system of record Meta Object Entity Name Entity Type Entity Definition Entity Scope Entity Active Ind Entity Logical Business Rule Logical Application Name Entity Synonym Name Logical Attribute Logical Attribute Definition Attribute Logical Business Rule Attribute Logical FK Ind Attribute Business Area Logical Business Function Data Subject Area Physical Column Name Physical Column Data Type Physical Column Length Physical Column Precision Strategic Modeling Tool Tactical Modeling Tool C C C C C U U C U DBMS Tool Data Integration Tool Reporting Tool U C C C C U C U C C U U C C U U C U C U C U C U (Cont.) 283 284 Appendix Table 1 Metadata object system of record (continued) Meta Object Strategic Modeling Tool Physical Column Decimal places Physical Column Default Value Physical Column Nullable Ind Physical Column Comment Physical Column Primary Key Ind Physical Column Foreign Key Ind Table Name Table Owner Table Type Table Comments Physical Table Name Physical Column Name Physical View Name Physical Database Name Physical Schema Name ETL Object Name Source Table Source Column Target Table Target Column ETL Job Name ETL Transformation Rule ETL Job Run Date ETL Job Execution Time ETL Job Row Count ETL Job Status Report Name Report Element Name Report Table Name Report Database Name Report DB Sequence Report Element Business Rule Tactical Modeling Tool DBMS Tool C U C U C U C U C C C C C U U U U U C C C C C Data Integration Tool Reporting Tool C R R R R C C C C C C C R R R R C Appendix 285 Metadata Usage Matrix Example The following table summarizes the metadata objects and the anticipated usage of each object in the following functions: Table 1 Summary of metadata objects usage Source – Metadata Object Entity Name Entity Type Entity Definition Entity Scope Entity Container Entity Active Ind Entity Logical Business Rule Logical Application Name Entity Synonym Name Logical Attribute Logical Attribute Definition Attribute Logical Business Rule Attribute Logical FK Ind Attribute Business Area Logical Business Function Data Subject Area Physical Column Name Physical Column Data Type Physical Column Length Physical Column Precision Physical Column Decimal Places Physical Column Default Value Physical Column Nullable Ind Data Lineage Impact Analysis Definition and or Glossary Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y (Cont.) 286 Appendix Table 1 Summary of metadata objects usage (continued) Source – Metadata Object Physical Column Comment Physical Column Primary Key Ind Physical Column Foreign Key Ind Table Name Table Owner Table Type Table Comments Physical Table Name Physical Column Name Physical View Name Physical Database Name Physical Schema Name ETL Object Name Source Table Source Column Target Table Target Column ETL Job Name ETL Transformation Rule ETL Job Run Date ETL Job Execution Time ETL Job Row Count ETL Job Status Report Name Report Element Name Report Table Name Report Database Name Report DB Sequence Report Element Business Rule Data Lineage Impact Analysis Definition and or Glossary Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Index Abstraction, linking structured and unstructured data, 230–231 Accuracy, metadata information, 33–34 Administration, infrastructure issues functionality requirements, 169 history keeping, 169–170 BI, see Business intelligence Broader term, definition component, 61, 63–64 Business Glossary features, 111–112 integrated technical and business metadata delivery, 152–153 Business intelligence (BI), business metadata delivery infrastructure, 163 Business metadata capture, see Capture, business metadata components, 13 definition, 38 delivery, see Delivery, business metadata funding, see Funding, business metadata historical perspective, 3–11 importance, 274–275 locations corporate forms, 15–16 reports, 14, 29–31 screens, 13–14 origins, 19–20 repository construction, see Metadata project resources, 281–282 technical metadata comparison, 12–13 conversion, 135–136, 182–186 infrastructure for integration, 165–166 separation, 140 tracking over time, 20–21 types, 158–160 Business rules business management, 238–239 business metadata, 237, 245 capturing rationale, 238–239 definition, 235–236 maintenance, 242 management, 243 metadata repository, 244–245 ruleflow, 242–243 sources, 237–238 systems, 239, 242–243 Call center volume, search problem quantification, 70 Capability Maturity Model Integration (CMMI), note-taking as asset producing, 265 Capture, business metadata barriers, 275 corporate knowledge base, 93–94 culture, 95–96 editing automation, 128–129 expansion of definition and descriptions, 129–131 granularization, 129 homonym resolution, 132–134 manual editing, 134–135 staging area, 134 synonym resolution, 131–132 Governance Lite™, 107–109, 111 individual documentation problem, 114–115 knowledge socialization, see Knowledge socialization metadata sources comparison of sources, 127 data warehouse, 126 database management system system catalogs, 124 documents, 123 enterprise resource planning applications, 122 extract-transform-load, 124–125 legacy systems, 125–126 on-line analytical processing tools, 124 on-line transaction processing, 125–126 reports, 122–123 spreadsheets, 123 principles, 95–96 publicity, 112–113 rationale, 90–93 technical metadata conversion to business metadata, 135–136 287 288 Index Capture, business metadata (Continued) technology search, 109–111 Web 2.0 folksonomy, 118–119 mashups, 115–116 overview, 115 Card catalog, see Library card catalog CDC, see Centers for Disease Control Centers for Disease Control (CDC), linking structured and unstructured data, 231 CIF, see Corporate Information Factory C-map, see Concept Map CMMI, see Capability Maturity Model Integration Collective intelligence, knowledge socialization, 97 Communications audits, 251 clarity problems bad business decisions, 57 English language limitations, 59 everyday communications, 56–57 faulty rollups, 57–58 units of measure differences, 59 classification, see Taxonomy definitions components, 61–62 guidelines, 60–61 importance, 59–60 miscellaneous guidelines, 64 usage notes, 62–64 historic library creation, 253 human/computer communication problem, 276–278 screening, 251–253 search problem information and knowledge workers, 65–66 information provider guidelines, 71–72 quantification, 67–70 search techniques, 71 tracking down information, 66–67 Compliance communications audits, 251 historic library creation, 253 screening, 251–253 data profiling, 254–256 financial audit metadata utility, 250 transaction background activities, 250–251 prospects, 280 Sarbanes-Oxley Act provisions, 248–240 types, 249–251 Concept Map (C-map), semantic framework, 205, 209 Conceptual model, semantic framework, 204 Controlled vocabulary (CV), semantic framework, 200–201 Corporate forms, business metadata content, 15–16 Corporate Information Factory (CIF), implementation, 81–82 Corporate knowledge base, components, 93–94 Create Read Update Delete (CRUD), conflict resolution, 167 CRM, see Customer relationship management Cross selling, business metadata capture rationale, 92 CRUD, see Create Read Update Delete Customer relationship management (CRM), business metadata capture rationale, 92 Customer, definition, 56 CV, see Controlled vocabulary DASD, see Direct access storage device Data, definition, 176 Database management system (DBMS) historical perspective, 5 metadata storage, 19 system catalog as metadata resource, 124 Data Flux, data quality presentation, 190–191 Data Governance Council, metadata stewardship, 42–43 Data quality continuum, 190 Data Warehousing report, 49 definition, 177 presentation, 189–190 Data Stewardship Council, metadata stewardship, 43–44 Data warehouse historical perspective, 7–9 infrastructure, 160–161 metadata resource, 126 metadata warehouse features, 161–162 DBMS, see Database management system Decision table, business rule representation, 239–242 Decision tree, business rule representation, 239, 241 Index Definition components, 61–62 dictionary role in information quality, 186 guidelines, 60–61 importance, 59–60 miscellaneous guidelines, 64 usage notes, 62–64 Delivery, business metadata examples corporate dictionary, 147–148 integrated technical and business metadata delivery, 152–153 mashups, 149–150 technical use, 151–152 training, 148–149 visual analytic techniques, 150–151 indirect usage accessibility from multiple places, 142–143 application access, 147 interactive reports, 145–146 overview, 141–142 Web delivery, 143–144 information quality business metadata, 188–190 infrastructure considerations business intelligence environments, 163 graphical affinity, 163–164 legacy environment, 162–163 mashups, 164 principles, 140–141 prospects, 280 Description Logics (DL), semantic framework, 206 Dictionary, see Glossary Direct access storage device (DASD), historical perspective, 4–6 Disk storage, historical perspective, 4–6 DL, see Description Logics Documents, metadata resource, 123 Editing, metadata automation, 128–129 expansion of definition and descriptions, 129–131 granularization, 129 homonym resolution, 132–134 manual editing, 134–135 staging area, 134 synonym resolution, 131–132 Employee turnover, business metadata capture rationale, 91–92 289 Enterprise resource planning (ERP), metadata resource, 85, 122 Entity/relationship (ER) model, semantic framework, 203–204, 208, 210–211 ER model, see Entity/relationship model ERP, see Enterprise resource planning ETL, see Extract-transform-load Extract-transform-load (ETL), metadata resource, 85, 124–125 Federated metadata, integrated metadata management, 168 Financial audit metadata utility, 250 transaction background activities, 250–251 First-order logic (FOL), semantic framework, 206 FOL, see First-order logic Folksonomy knowledge capture, 118–119 self-organizing tags, 77 Forms, see Corporate forms Fourth generation language historical perspective, 6–7 metadata handling, 11 Funding, business metadata advantages and disadvantages of approaches, 52–53 centralized implementation, 51 localized implementation, 51–52 overview, 50–51 Glossary business functions, 60 information quality role, 186 semantic framework, 201 Governance Lite™, knowledge capture, 107–109, 111 Granularization, metadata, 129 Graphical affinity, business metadata delivery infrastructure, 163–164 Grid, metadata representation, 18 Groupware, knowledge socialization, 100–103, 279 Homonyms, resolution, 132–134 Industrial recognition, text, 227 Information quality business and technical metadata interaction, 177–186 business metadata delivery, 188–190 290 Index Information quality (Continued) definition, 177 dictionary role, 186 methodology, 187–188 Information Technology (IT) department challenges, 278–279 metadata responsibility, 38–39 Information, definition, 177 IT department, see Information Technology department KB, see Knowledge base KM, see Knowledge management Knowledge base (KB) definition, 267 building, 267 Knowledge management (KM) business metadata intersection artifact generation, 262–263 corporate dictionary example, 263 definition, 260 goals, 260–261 importance, 261 social issues graying work force, 269–270 socialization effect on knowledge, 270 tacit knowledge, see Tacit knowledge techniques, 267–268 Knowledge socialization collective intelligence, 97 experts, 97–98 groupware, 100–103, 279 knowledge management, 268, 270 technology fostering portal and collaboration servers, 100–103 social networking, 99 wikis, 103–106 Knowledge worker metadata capture, 94 search problem, 65–66 Legacy systems business metadata delivery infrastructure, 162–163 metadata resource, 125–126 Library card catalog, metadata analogy, 27–29 Life cycle, metadata, 45–48 Magnetic tape data storage, 4 languages for data reading, 10 Mashup business metadata delivery, 149–150, 164 knowledge capture, 115–116 Master data management (MDM) conflict resolution, 167 overview, 22 MDM, see Master data management Metadata definition, 9, 26–27 examples, 9 grid representation, 18 management importance, 32 metamodel, 158–160 system of record example, 283–284 usage matrix example, 285–286 Metadata project business metadata versus technical metadata, 83–84 buying versus building, 170–172 classification, 82–83 funding, see Funding, business metadata iterations of development, 84 local metadata tools, 85 metadata sources, 86–87 preexisting repositories, 172–173 rationale, 80–82 scope defining, 85–87 Metadata Stewardship Council, responsibilities, 44–45 Narrower term, definition component, 61, 63–64 National Cancer Institute (NCI), semantic vocabulary implementation, 214–216 NCI, see National Cancer Institute Null, data profiling, 183, 185–186 ODS, see Operational data store OLAP, see Online analytical processing On-line analytical processing, metadata resource, 85124 On-line transaction processing, metadata resource, 125–126 Ontology, semantic framework, 207 Operational data store (ODS), data warehousing, 8 Opportunity cost, search problem quantification, 69 OWL, see Web Ontology Language Ownership, definition, 40 Patterns, identification, 183–184, 186 PC, see Personal computer Index Personal computer (PC) historical perspective, 7 metadata handling, 11 Preferred term, definition component, 62 Punch cards historical perspective, 4 metadata, 9–10 Quality, see Data quality; Information quality Range of values, data profiling, 183, 186 RDF, see Resource Definition Framework Reference file overview, 21–22 updating, 22 Regulations, see Compliance Related term, definition component, 61 Reports interactive reports, 145–146 metadata resource, 14, 29–31, 122–123 Repository, see Metadata project Resource Definition Framework (RDF), semantic framework, 204–205 Reuse, metadata, 32–33 Sales, search problem quantification, 70 Sarbanes-Oxley Act, provisions, 248–240 Screen, business metadata content, 13–14 Search problem enterprise search, 279 information and knowledge workers, 65–66 information provider guidelines, 71–72 quantification, 67–70 search techniques, 71 tracking down information, 66–67 Self-organizing map, linking structured and unstructured data, 233 Self-organizing tags, taxonomy, 77 Semantics business metadata delivering definitions and relationships, 208–209 exposing semantics to business, 210–211 expression, 209–210 overview, 207–208 context sensitivity, 197–199 framework Concept Map, 205, 209 conceptual model, 204 controlled vocabulary, 200–201 Description Logics, 206 291 entity/relationship model, 203–204, 208, 210–211 first-order logic, 206 glossary, 201 ontology, 207 Resource Definition Framework, 204–205 taxonomy, 202 thesauri, 203 topic map, 205 UML, 205–206 Web Ontology Language, 204–205, 210 human/computer concept, 199–200, 278 importance, 196–197 practical issues integration, 211–212 National Cancer Institute semantic vocabulary implementation, 214–216 service-oriented architecture, 214 Web Services, 212–213 prospects for integration and discovery, 280 semantic Web, 195–196 spectrum, 200–201, 208 Semistructured data, examples and technologies, 222–223 Serial transfer, knowledge management, 267–268 Service-oriented architecture (SOA) integrated metadata management, 168 semantics integration, 214 SOA, see Service-oriented architecture SOAP, semantic interface, 212 Socialization, see Knowledge socialization Social networking, knowledge socialization, 99 Spreadsheets, metadata resource, 123 Stemmed words, text distillation, 225 Stewardship business metadata approaches, 44–45 artifacts, 44 Data Governance Council, 42–43 Data Stewardship Council, 43–44 historical perspective, 41–42 Metadata Stewardship Council, 44–45 definition, 40–41 Structured metadata characteristics, 16–18 examples and technologies, 219–220 linking structured and unstructured data abstraction, 230–231 examples, 231, 233 292 Index Structured metadata (Continued) integration, 230 unstructured data comparison, 221 Synonyms, resolution, 131–132 Tacit knowledge definition, 94, 264 note-taking as asset producing, 265–266 transfer nurturing, 266 Taxonomy basic rules, 73, 75 document categorization, 76 governance and taxonomy, 77 language and vocabulary, 75–76 lowest common denominator, 75 overview, 72 self-organizing tags, 77 semantic framework, 202 simplicity, 76 Team Room, knowledge socialization, 100–103 Technical metadata business metadata comparison, 12–13 conversion, 135–136 infrastructure for integration, 165–166 separation, 140 categories, 2 Technical metadata conversion to business metadata, 135–136, 182–186 profiling, 179–182 Text business metadata terms, 227–228 communications audits, 252–253 distillation, 224–229 extraneous words, 225 industrial recognition, 227 pulling, 223 relationship recognition, 228–229 stemmed words, 225 word counting, 226 Thesauri, semantic framework, 203 Topic map, semantic framework, 205 UML, semantic framework, 205–206 Unstructured metadata characteristics, 16–18 examples and technologies, 220–221 mining prospects, 281 structured data comparison, 221 text business metadata terms, 227–228 distillation, 224–229 extraneous words, 225 industrial recognition, 227 pulling, 223 relationship recognition, 228–229 stemmed words, 225 word counting, 226 Value/frequency report, data profiling, 181, 184–186 Web 2.0 knowledge capture folksonomy, 118–119 mashups, 115–116 overview, 115 semantic Web, 195–196 Web Ontology Language (OWL), semantic framework, 204–205, 210 Web Services, semantics interface, 212–213 Wiki governance, 106 knowledge capture, 104 limitations, 105–106 portal collaboration comparison, 105 wikinomics, 104 Wikipedia, 103–104, 212 Words, see Text
Business Metadata Praise for Business Metadata “Despite the presence of some excellent books on what is essentially “technical” metadata, up until now there has been a dearth of wellpresented material to help address the growing need for interaction at the conceptual and semantic levels between data professionals and the business clients they support. In Business Metadata, Bill, Bonnie, and Lowell provide the means for bridging the gap between the sometimes “fuzzy” human perception of data that fuels business processes and the rigid information management models used by business applications. Look to the future: next generation business intelligence, enterprise content management and search, the semantic web all will depend on business metadata. Read this book!” —David Loshin, President, Knowledge Integrity Incorporated These authors have written a book that ventures into new territory for data and information management. There are several books about metadata, but this is the first to offer in-depth discussion of the important topic of business metadata.
Data Mining: Concepts and Techniques: Concepts and Techniques by Jiawei Han, Micheline Kamber, Jian Pei
bioinformatics, business intelligence, business process, Claude Shannon: information theory, cloud computing, computer vision, correlation coefficient, cyber-physical system, database schema, discrete time, distributed generation, finite state, information retrieval, iterative process, knowledge worker, linked data, natural language processing, Netflix Prize, Occam's razor, pattern recognition, performance metric, phenotype, random walk, recommendation engine, RFID, semantic web, sentiment analysis, speech recognition, statistical model, stochastic process, supply-chain management, text mining, thinkpad, Thomas Bayes, web application
To demonstrate the importance of applications as a major dimension in data mining research and development, we briefly discuss two highly successful and popular application examples of data mining: business intelligence and search engines. 1.6.1. Business Intelligence It is critical for businesses to acquire a better understanding of the commercial context of their organization, such as their customers, the market, supply and resources, and competitors. Business intelligence (BI) technologies provide historical, current, and predictive views of business operations. Examples include reporting, online analytical processing, business performance management, competitive intelligence, benchmarking, and predictive analytics. “How important is business intelligence?” Without data mining, many businesses may not be able to perform effective market analysis, compare customer feedback on similar products, discover the strengths and weaknesses of their competitors, retain highly valuable customers, and make smart business decisions.
Without data mining, many businesses may not be able to perform effective market analysis, compare customer feedback on similar products, discover the strengths and weaknesses of their competitors, retain highly valuable customers, and make smart business decisions. Clearly, data mining is the core of business intelligence. Online analytical processing tools in business intelligence rely on data warehousing and multidimensional data mining. Classification and prediction techniques are the core of predictive analytics in business intelligence, for which there are many applications in analyzing markets, supplies, and sales. Moreover, clustering plays a central role in customer relationship management, which groups customers based on their similarities. Using characterization mining techniques, we can better understand features of each customer group and develop customized customer reward programs. 1.6.2.
In this context, different clustering methods may generate different clusterings on the same data set. The partitioning is not performed by humans, but by the clustering algorithm. Hence, clustering is useful in that it can lead to the discovery of previously unknown groups within the data. Cluster analysis has been widely used in many applications such as business intelligence, image pattern recognition, Web search, biology, and security. In business intelligence, clustering can be used to organize a large number of customers into groups, where customers within a group share strong similar characteristics. This facilitates the development of business strategies for enhanced customer relationship management. Moreover, consider a consultant company with a large number of projects. To improve project management, clustering can be applied to partition projects into categories based on similarity so that project auditing and diagnosis (to improve project delivery and outcomes) can be conducted effectively.
Keeping Up With the Quants: Your Guide to Understanding and Using Analytics by Thomas H. Davenport, Jinho Kim
Black-Scholes formula, business intelligence, business process, call centre, computer age, correlation coefficient, correlation does not imply causation, Credit Default Swap, en.wikipedia.org, feminist movement, Florence Nightingale: pie chart, forensic accounting, global supply chain, Hans Rosling, hypertext link, invention of the telescope, inventory management, Jeff Bezos, margin call, Moneyball by Michael Lewis explains big data, Myron Scholes, Netflix Prize, p-value, performance metric, publish or perish, quantitative hedge fund, random walk, Renaissance Technologies, Robert Shiller, Robert Shiller, self-driving car, sentiment analysis, six sigma, Skype, statistical model, supply-chain management, text mining, the scientific method
If you don’t have a lot of mathematical or statistical skills, it may also be the stage that is most likely to be entrusted to others who do have the quantitative abilities you need (see “How to Find a Quant”). But whether or not you’re actually crunching the numbers yourself, it’s useful to know something about the three steps involved in solving the problem. * * * How to Find a Quant If you need a quantitative analyst to help you solve your problem, here are some ways to do it: If you work for a large company, you probably have some—so look in places like Market Research, Business Intelligence, or Operations Research. If you don’t have any, there are plenty of consultants to whom you can turn. Do an Internet search for “business analytics consultants” or look at the helpful list from KDnuggets (http://www .kdnuggets.com/companies/consulting.html). If you want some offshore analytics consultants, the best are to be found in India; check out such firms as Mu Sigma, Fractal Analytics, and Genpact.
For that approach, you will need software that focuses primarily on reporting. Dashboards, scorecards, and alerts are all forms of reporting. In “Key Software Vendors for Different Analysis Types,” we list some of the key vendors for reporting software. 5. Data analysis * * * Key Software Vendors for Different Analysis Types (listed alphabetically) REPORTING SOFTWARE BOARD International IBM Cognos Information Builders WebFOCUS Oracle Business Intelligence (including Hyperion) Microsoft Excel/SQL Server/SharePoint MicroStrategy Panorama SAP BusinessObjects INTERACTIVE VISUAL ANALYTICS QlikTech QlikView Tableau TIBCO Spotfire QUANTITATIVE OR STATISTICAL MODELING IBM SPSS R (an open-source software package) SAS * * * While all of the listed reporting software vendors also have capabilities for graphical display, some vendors focus specifically on interactive visual analytics, or the use of visual representations of data and reporting.
Considering that employers’ demand for MSA graduates has been increasing, it is natural that this type of degree program will be opened at other universities as well. SAS has sponsored new programs at Louisiana State and Texas A&M. The University of San Francisco has created a West Coast program as well, and New York University has begun one in New York. One recent survey found that fifty-nine universities were offering either degrees or majors in business analytics or business intelligence—thirty-seven at the masters level and twenty-two undergraduate programs.13 Schools are also beginning to offer courses in data science, and degree programs will follow soon. Quantitative Habits Attitudes are important, but so are habits. It has often been said that it is easier to act your way into a new way of thinking than to think your way into a new way of acting. If you practice the following quantitative habits, you’ll eventually find that you’ve developed a quantitative attitude.
The Start-Up of You by Reid Hoffman
Airbnb, Andy Kessler, Black Swan, business intelligence, Cal Newport, Clayton Christensen, commoditize, David Brooks, Donald Trump, en.wikipedia.org, fear of failure, follow your passion, future of work, game design, Jeff Bezos, job automation, late fees, Marc Andreessen, Mark Zuckerberg, Menlo Park, out of africa, Paul Graham, Peter Thiel, recommendation engine, Richard Bolles, risk tolerance, rolodex, shareholder value, side project, Silicon Valley, Silicon Valley startup, social web, Steve Jobs, Steve Wozniak, Tony Hsieh, transaction costs
NAVIGATE PROFESSIONAL CHALLENGES WITH NETWORK INTELLIGENCE Entrepreneurs navigate the day-to-day issues of running a company by gathering intelligence: actionable, timely information on all facets of their business, including industry trends, opportunities, competitors’ activities, customer sentiment, promising young talent, and sales trends. In a business, intelligence serves as a GPS device. You need good intelligence to run the start-up of you. The preceding chapters should have prompted questions in your mind like: How desirable are my skills in the changing market? How do I know when I should pivot into a new industry niche? What are the best job opportunities and how do I exploit them? These are not easy questions. They’re certainly not questions you can answer by merely reflecting for a few minutes or filling out a worksheet. You, too, need business intelligence to navigate these challenges. You get it by talking to people in your network. It’s people who help you understand your assets, aspirations, and the market realities; it’s people who help you vet and get introduced to possible allies and trust connections; it’s people who help you track the risk attached to a given opportunity.
Entrepreneurs deal with these uncertainties, changes, and constraints head-on. They take stock of their assets, aspirations, and the market realities to develop a competitive advantage. They craft flexible, iterative plans. They build a network of relationships throughout their industy that outlives their start-up. They aggressively seek and create breakout opportunities that involve focused risk, and actively manage that risk. They tap their network for the business intelligence to navigate tough challenges. And, they do these things from the moment they hatch that nascent idea to every day after that—even as the companies go from being run out of a garage to occupying floors of office space. To succeed professionally in today’s world, you need to adopt these same entrepreneurial strategies. They are valuable no matter your career stage. They are urgent whether you’re just out of college, a decade into the workforce and angling for that next big move, or launching a brand-new career later in life.
European Founders at Work by Pedro Gairifo Santos
business intelligence, cloud computing, crowdsourcing, fear of failure, full text search, information retrieval, inventory management, iterative process, Jeff Bezos, Lean Startup, Mark Zuckerberg, natural language processing, pattern recognition, pre–internet, recommendation engine, Richard Stallman, Silicon Valley, Skype, slashdot, Steve Jobs, Steve Wozniak, subscription business, technology bubble, web application, Y Combinator
Everyone was in some sort of cost-cutting mode and with the business intelligence that Business Objects offered, customers could optimize their costs and see where the inefficiencies lay. So, we switched our value proposition from “make your business better, add visibility and increase revenue” to “make your business more efficient, see where you waste money and cut your costs.” That alteration enabled us to keep selling in a difficult economy. Santos: So, from that period on, to the actual period that Business Objects was sold, can you walk us through a bit of that period? Liautaud: We acquired Crystal in 2003, integrated the business in 2004, and reached a billion dollars in revenue in 2005. We continued to expand and bought a few small businesses. By 2007 we saw about $1.5 billion in revenue. Business intelligence was becoming more and more important to the IT industry and a number of the industry's biggest companies realized that they needed to be in it.
I knew we had to make some critical decisions. First of all, we needed to shift the centre of gravity of the company from France to the US. At that time, the company's management team was in Paris but I decided to move our senior executives, along with a new CFO, to California to be closer to our customers and the US financial community. I also restructured the software development process, created a brand new internet-based business intelligence product and, over the next year and a half, we fixed the buggy product line which, in the end, was very attractive to customers. We started to grow again and over two years, our market cap went from $100 million to almost $5 billion. 1997 to 1999 was an amazing time for the company as we expanded into more and more countries. When the technology bubble burst, we emerged relatively unscathed because by then we were a large, stable, profitable company with thousands of customers.
US, 235 France, 234–235 freelancer relationship, 240 LinkedIn, 226 next generation founders, 238–239 online community creation, 230–231 pivot board, 234 problems and solution, 228–230 RapidSite, 226 Silicon Valley, 230, 238 social networking campaign, 227 technology evolution, 231–232 Twitter, 233, 236 Ublog, 226 US lessons, 227–228 US vs. France business, 239–240 venture capital fund raise, 235 Wi-Fi sharing software, Skype, 233 WordPress, 226 Liautaud, Bernard. Business Objects, 49 Atlas Ventures, 50 business intelligence and analytics, 55 California, marketing head, 53 Coface, 50 cost-cutting mode, 54 Crystal Reports, 52, 53 entrepreneur needs, 56 European investors, 54 European market, 51 European vs. US venture, 56 Goldman Sachs, 51 Innovacom, France Telecom, 50 Michel (Jean), Oracle interface, 49 Schwartz (John), CEO, 55–56 key lessons, success, 51 NASDAQ, 50 near-death experience, 53 new hire risk, 51 Partech International, 50 product marketing, Oracle, 49 SAP deal, 55 strong execution model, 57 US subsidiary and French company, internal issues, 52 M, N, O Moross, Richard.
Industry 4.0: The Industrial Internet of Things by Alasdair Gilchrist
3D printing, additive manufacturing, Amazon Web Services, augmented reality, autonomous vehicles, barriers to entry, business intelligence, business process, chief data officer, cloud computing, connected car, cyber-physical system, deindustrialization, fault tolerance, global value chain, Google Glasses, hiring and firing, industrial robot, inflight wifi, Infrastructure as a Service, Internet of things, inventory management, job automation, low skilled workers, millennium bug, pattern recognition, peer-to-peer, platform as a service, pre–internet, race to the bottom, RFID, Skype, smart cities, smart grid, smart meter, smart transportation, software as a service, stealth mode startup, supply-chain management, trade route, web application, WebRTC, WebSocket, Y2K
It will be possible to manage threats proactively by moving cargo from air to sea or vice versa to mitigate the threats of strike action. Similarly, urgent cargo routes can be altered in real-time if predicative analysis of all the global data shows a high risk of civil unrest or bad weather on route, which could seriously delay delivery. Predictive analysis through Big Data is becoming a required tool for business intelligence analysis; it is believed that over 80% of businesses will adopt it in one form or another in 2016. The promise that predictive analysis holds for global logistics is that they will be able to take proactive action to mitigate potential threats to their operations and keep their freight moving to the destination. Retail Retailers, like most businesses, suffer IT costs and overheads, which directly affects their profits, as they must pass these costs onto the customer as part of the cost of goods.
Say you contract an offshore company to produce one million shirts. Now that they have the template, what is stopping them from running off 2 million extras to sell on the black market? 55 56 Chapter 3 |TheTechnical and Business Innovators of the Industrial Internet The big point about Big Data is that it requires vast amounts of intellect to distil business value from it. Creating data lakes will not automatically facilitate business intelligence. If you do not know the correct question to ask of the data, how you can expect a sensible answer? This is where we have to understand how or if machines think and collaborate. M2M Learning and Artificial Intelligence Big Data empowers M2M learning and artificial intelligence, the larger the pool of data the more trustworthy the forecasts—or so it would seem. M2M learning is very important and sometimes very simple, for example a multiple-choice exam.
Previously, interest in advanced algorithms had been limited to certain business sectors such as insurance, marketing, and finance where risk and opportunity were the key drivers. However, with the advent of Big Data and the cloud resources to leverage it, has come interest from many other sectors in industry, in search of ways to improve decisionmaking, reduce risk, and optimize business outcomes. This has brought about a seismic shift away from traditional business intelligence, which focuses more on descriptive analysis that uses historical data to determine what has happened. Instead, business leaders are looking to advanced analytics, which complements descriptive analysis by taking the analysis further via diagnostic analysis by asking, not just what happened, but why it happened. Furthermore, it uses predictive analysis to ask what will happen. And finally, it uses prescriptive analytics to ask what you should do (see Figure 4-7).
Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Apple's 1984 Super Bowl advert, backtesting, Black Swan, book scanning, bounce rate, business intelligence, business process, call centre, commoditize, computer age, conceptual framework, correlation does not imply causation, crowdsourcing, dark matter, data is the new oil, en.wikipedia.org, Erik Brynjolfsson, Everything should be made as simple as possible, experimental subject, Google Glasses, happiness index / gross national happiness, job satisfaction, Johann Wolfgang von Goethe, lifelogging, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, mass immigration, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, Norbert Wiener, personalized medicine, placebo effect, prediction markets, Ray Kurzweil, recommendation engine, risk-adjusted returns, Ronald Coase, Search for Extraterrestrial Intelligence, self-driving car, sentiment analysis, software as a service, speech recognition, statistical model, Steven Levy, text mining, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Turing test, Watson beat the top human players on Jeopardy!, X Prize, Yogi Berra, zero-sum game
Eric advocates quite clearly how some choices are predictably more profitable than others—and I agree!” —Dennis R. Mortensen, CEO of Visual Revenue, former Director of Data Insights at Yahoo! “This book is an invaluable contribution to predictive analytics. Eric’s explanation of how to anticipate future events is thought provoking and a great read for everyone.” —Jean Paul Isson, Global VP Business Intelligence and Predictive Analytics, Monster Worldwide; coauthor, Win with Advanced Business Analytics: Creating Business Value from Your Data “Eric Siegel’s book succeeds where others have failed—by demystifying big data and providing real-world examples of how organizations are leveraging the power of predictive analytics to drive measurable change.” —Jon Francis, Senior Data Scientist, Nike “Predictive analytics is the key to unlocking new value at a previously unimaginable economic scale.
—Jared Waxman, Web Marketer at LegalZoom, previously at Adobe, Amazon, and Intuit “Siegel covers predictive analytics from start to finish, bringing it to life and leaving you wanting more.” —Brian Seeley, Manager, Risk Analytics, Paychex, Inc. “A wonderful look into the world of predictive analytics from the perspective of a true practitioner.” —Shawn Hushman, VP, Analytic Insights, Kelley Blue Book “An excellent exposition on the next generation of business intelligence—it’s really mankind’s latest quest for artificial intelligence.” —Christopher Hornick, President and CEO, HBSC Strategic Services “A must—Predictive Analytics provides an amazing view of the analytical models that predict and influence our lives on a daily basis. Siegel makes it a breeze to understand, for all readers.” —Zhou Yu, Online-to-Store Analyst, Google “[Predictive Analytics is] an engaging, humorous introduction to the world of the data scientist.
Whereas forecasting estimates the total number of ice cream cones to be purchased next month in Nebraska, predictive technology tells you which individual Nebraskans are most likely to be seen with cone in hand. PA leads within the growing trend to make decisions more “data driven,” relying less on one’s “gut” and more on hard, empirical evidence. Enter this fact-based domain and you’ll be attacked by buzzwords, including analytics, big data, business intelligence, and data science. While PA fits underneath each of these umbrellas, these evocative terms refer more to the culture and general skill sets of technologists who do an assortment of creative, innovative things with data, rather than alluding to any specific technology or method. These areas are broad; in some cases, they refer simply to standard Excel reports—that is, to things that are important and require a great deal of craft, but may not rely on science or sophisticated math.
The Invisible Web: Uncovering Information Sources Search Engines Can't See by Gary Price, Chris Sherman, Danny Sullivan
AltaVista, American Society of Civil Engineers: Report Card, bioinformatics, Brewster Kahle, business intelligence, dark matter, Donald Davies, Douglas Engelbart, Douglas Engelbart, full text search, HyperCard, hypertext link, information retrieval, Internet Archive, joint-stock company, knowledge worker, natural language processing, pre–internet, profit motive, publish or perish, search engine result page, side project, Silicon Valley, speech recognition, stealth mode startup, Ted Nelson, Vannevar Bush, web application
This book introduces readers to the basics of interface design and explains why a design evaluation should be tied to the use and purchase of information resources. 1999/224 pp/softbound/ISBN 0-910965-31-5 $29.95 1999/224 pp/hardbound/ISBN 0-910965-39-0 $39.95 397 448 Internet Business Intelligence How to Build a Big Company System on a Small Company Budget By David Vine According to author David Vine, business success in the competitive, global marketplace of the 21st century will depend on a firm’s ability to use information effectively—and the most successful firms will be those that harness the Internet to create and maintain a powerful information edge. In Internet Business Intelligence, Vine explains how any company can build a complete, low-cost, Internet-based business intelligence system that really works. If you’re fed up with Internet hype and wondering “Where’s the beef?,” you’ll appreciate this savvy, nononsense approach to using the Internet to solve everyday business problems and stay one step ahead of the competition. 2000/448 pp/softbound/ISBN 0-910965-35-8 $29.95 net.people The Personalities and Passions Behind the Web Sites By Eric C.
This book helps readers identify overseas buyers, find foreign suppliers, investigate potential partners and competitors, uncover international market research and industry analysis, and much more. 2001/380 pp/softbound/ISBN 0-910965-46-3 $29.95 442 Millennium Intelligence Understanding and Conducting Competitive Intelligence in the Digital Age By Jerry P. Miller and the Business Intelligence Braintrust With contributions from 12 of the world’s leading business intelligence practitioners, here is a tremendously informative and practical look at the CI process, how it is changing, and how it can be managed effectively in the Digital Age. Loaded with case studies, tips, and techniques. 2000/276 pp/softbound/ISBN 0-910965-28-5 $29.95 Internet Prophets The Complete Guide to Enlightened E-Business Strategies By Mary Diffley Since the bursting of the dot.com balloon, companies are approaching e-business with a new wariness—and rightly so, according to author and entrepreneur Mary Diffley.
Only Humans Need Apply: Winners and Losers in the Age of Smart Machines by Thomas H. Davenport, Julia Kirby
AI winter, Andy Kessler, artificial general intelligence, asset allocation, Automated Insights, autonomous vehicles, basic income, Baxter: Rethink Robotics, business intelligence, business process, call centre, carbon-based life, Clayton Christensen, clockwork universe, commoditize, conceptual framework, dark matter, David Brooks, deliberate practice, deskilling, digital map, Douglas Engelbart, Edward Lloyd's coffeehouse, Elon Musk, Erik Brynjolfsson, estate planning, fixed income, follow your passion, Frank Levy and Richard Murnane: The New Division of Labor, Freestyle chess, game design, general-purpose programming language, Google Glasses, Hans Lippershey, haute cuisine, income inequality, index fund, industrial robot, information retrieval, intermodal, Internet of things, inventory management, Isaac Newton, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Khan Academy, knowledge worker, labor-force participation, lifelogging, loss aversion, Mark Zuckerberg, Narrative Science, natural language processing, Norbert Wiener, nuclear winter, pattern recognition, performance metric, Peter Thiel, precariat, quantitative trading / quantitative ﬁnance, Ray Kurzweil, Richard Feynman, Richard Feynman, risk tolerance, Robert Shiller, Robert Shiller, Rodney Brooks, Second Machine Age, self-driving car, Silicon Valley, six sigma, Skype, speech recognition, spinning jenny, statistical model, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, superintelligent machines, supply-chain management, transaction costs, Tyler Cowen: Great Stagnation, Watson beat the top human players on Jeopardy!, Works Progress Administration, Zipcar
But a human was still required to create and interpret the analysis. That’s why, when you look at the matrix in Figure 3.1 you see in the upper-left corner a cell that represents the intersection of “human support” and “analyzing numbers.” When we map the progression of smart machines across the decades, the story starts here. Chances are, if you are a decision-maker in a large organization, you have found yourself working with business intelligence software, data visualization tools, and hypothesis-driven analytics. In the overall pattern by which automation comes to knowledge work, such tools represent square one. Unfolding from that point to the squares farthest from it on the horizontal is the history and future of smart machines, moving from the least to the greatest of their increasingly unnerving capabilities. Newer Ways to Support Humans We won’t march through every cell of this matrix; we trust you get the idea of it well enough to fill in some of the blanks in our narrative.
For health insurance giant UnitedHealthcare, she began to step up—not just looking at individual insurance policies, but forecasting the impact of new technologies and macroeconomic conditions on health-care costs. At Humana, another large firm (recently acquired by Aetna), Tourville continued her focus on modeling external trends, but she also led the company’s focus on analytics and data-driven decision-making. She was responsible there for the Business Intelligence and Informatics Competency Center, and had responsibility for $50 million of analytics-oriented projects and systems. At her current employer, Anthem—the second-largest company in the U.S. health insurance industry—she’s still stepping in and stepping up. She’s responsible for the “Commercial Health Care Economics” function, analyzing medical expense trends and understanding their movements, as well as the systems that report on such topics.
That means lots of reports, sliced and diced in different ways for different people. Aaron Kechley at DataXu told us that improving reporting has been a big deal for his company for several years now. He said, “We learned the hard way that you can’t preconceive what people will want in a report—people want to process information in different ways.” As a result, DataXu has been investing heavily in the latest “business intelligence” tools—new metrics, custom dashboards, and tailored reports. In fact, one of DataXu’s primary offerings these days is a tool that actually shows marketers whether their digital ads are working or not. This approach, which goes beyond reporting, has been done for a while with traditional statistical analysis. Now, however, DataXu has worked out a way to use rigorous and frequent—but still small—experiments to see if an ad or promotion is working.
Hacking Growth: How Today's Fastest-Growing Companies Drive Breakout Success by Sean Ellis, Morgan Brown
Airbnb, Amazon Web Services, barriers to entry, bounce rate, business intelligence, business process, correlation does not imply causation, crowdsourcing, DevOps, Elon Musk, game design, Google Glasses, Internet of things, inventory management, iterative process, Jeff Bezos, Khan Academy, Lean Startup, Lyft, Mark Zuckerberg, market design, minimum viable product, Network effects, Paul Graham, Peter Thiel, Ponzi scheme, recommendation engine, ride hailing / ride sharing, side project, Silicon Valley, Silicon Valley startup, Skype, Snapchat, software as a service, Steve Jobs, subscription business, Uber and Lyft, Uber for X, working poor, Y Combinator, young professional
Growth teams don’t necessarily replace more traditional departments, but rather complement them, and help them optimize their approaches. At early stage start-ups, avoiding these silos from the start is advised, but as a start-up grows, more traditional marketing groups can be established alongside a dedicated growth team. And at larger, established firms, teams can complement the existing product, marketing, engineering, and business intelligence groups, collaborating with them and helping to open up more effective communication across them. As Sean’s experience with Dropbox shows, the process can be implemented by even the smallest of teams, which for many start-ups, especially in the early growth phase, should be run by the founder and comprise the entire company. For larger firms that must contend with existing structures and cultures resistant to change, small teams can be set up independently and even for finite projects, like perhaps the launch of a new product or a specific marketing channel, such as mobile.
For example, at Netflix, by examining the movies and shows that customers were watching, the company found that Kevin Spacey films and political drama series were both hugely popular with their customers. That insight gave the company confidence to green-light the development of House of Cards, which became not only a huge hit, but also a must-have experience for many subscribers.27 Similarly, at RJMetrics, a business intelligence company, the team found that users who edited a chart in the software during their free trial period were twice as likely to convert to paying customers as those who didn’t and that that number went up even higher when a trial user edited two charts. So what did RJMetrics do? They made the editing of a chart a key step in their new user orientation.28 PIVOTING TO THE UNEXPECTED These distinctive behaviors and preferences can be hard to uncover, in part because sometimes they are so unexpected; paradoxically, you often don’t know what you’re looking for until you find it.
The selections are based on a combination of a customer’s search history and buying habits, and data about the habits of other shoppers like that customer. All Amazon shoppers in effect see their own version of Amazon with a unique experience tailored to their preferences. Some recommendation engines, such as Amazon’s, as well as those deployed by Google and Netflix, are incredibly complex, but many are based on relatively simple math. As Colin Zima, the chief analytics officer at Looker, a business intelligence software, explains, it can be relatively easy to generate recommendations based on a simple formula called a Jaccard index, or Jaccard similarity coefficient, which determines how similar two products are to each other. This helps to recommend additional items that a customer might want to buy because the software has calculated that the items, when purchased, are often purchased together.
The Silent Intelligence: The Internet of Things by Daniel Kellmereit, Daniel Obodovski
3D printing, Airbnb, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, autonomous vehicles, barriers to entry, business intelligence, call centre, Clayton Christensen, cloud computing, commoditize, connected car, crowdsourcing, data acquisition, en.wikipedia.org, Erik Brynjolfsson, first square of the chessboard, first square of the chessboard / second half of the chessboard, Freestyle chess, Google X / Alphabet X, Internet of things, lifelogging, Metcalfe’s law, Network effects, Paul Graham, Ray Kurzweil, RFID, Robert Metcalfe, self-driving car, Silicon Valley, smart cities, smart grid, software as a service, Steve Jobs, web application, Y Combinator, yield management
Maybe in certain domains or certain specific applications, but I think there’s a lot of opportunity, I mean a ton of opportunity there. I think on the software business analytics, it becomes easier and easier to build applications. To me, at the end of the day, M2M is really about data. You’re getting data off devices or sensors that have been hard to get, but once it’s in the cloud, it’s data. So all the data tools, if it’s big data or any kind of business intelligence software, all that stuff is applicable to it. So I think just the same tools and technologies, like Google or Facebook, really do a lot of analytics on their users, and the amount of data that they have, they’re very applicable into the M2M space. When deciding on which networks to run an M2M solution, we would like to point out the ubiquity versus granularity problem. For example, RFID tags are very low cost, don’t need a battery, and have an infinite life.
Since investment in the Internet of Things has until now been more of a futuristic topic, and the understanding and definition of the market varies notably, forecasts are all over the place. For example, IDC, a market research firm, estimates the value of intelligent systems at $1.7 trillion already, growing to $2.4 trillion by 2017.30 It’s interesting if you look at the high-growth markets that are currently developing around cloud computing, big data, and business intelligence. These markets are in the double-digit billions, and are often not counted toward the M2M market. This shows how blurry the borders are, and that we can expect a number of additional growth segments that we do not see or envision today. The question many investors raise is: How does this growth come about? In addition, what are the core growth segments? How much of the value will be captured by existing players?
Big Data Glossary by Pete Warden
It allows you to update the search index with much lower latency, has a more minimal REST/JSON-based interface and configuration options, and scales horizontally in a more seamless way. It doesn’t yet have the community or number of contributors of the more established project, though, and it is missing some of the broader features that Solr offers, so it’s worth evaluating both. Datameer Though it’s aimed at the well-known business intelligence market, Datameer is interesting because it uses Hadoop to power its processing. It offers a simplified programming environment for its operators to specify the kind of analysis they want, and then handles converting that into MapReduce jobs behind the scenes. It also has some user-friendly data importing tools, as well as visualization options. It’s a sign of where data processing solutions are headed, as we get better at building interfaces and moving to higher and more powerful abstraction levels.
Data Scientists at Work by Sebastian Gutierrez
Albert Einstein, algorithmic trading, Bayesian statistics, bioinformatics, bitcoin, business intelligence, chief data officer, clean water, cloud computing, commoditize, computer vision, continuous integration, correlation does not imply causation, creative destruction, crowdsourcing, data is the new oil, DevOps, domain-specific language, Donald Knuth, follow your passion, full text search, informal economy, information retrieval, Infrastructure as a Service, Intergovernmental Panel on Climate Change (IPCC), inventory management, iterative process, lifelogging, linked data, Mark Zuckerberg, microbiome, Moneyball by Michael Lewis explains big data, move fast and break things, move fast and break things, natural language processing, Network effects, nuclear winter, optical character recognition, pattern recognition, Paul Graham, personalized medicine, Peter Thiel, pre–internet, quantitative hedge fund, quantitative trading / quantitative ﬁnance, recommendation engine, Renaissance Technologies, Richard Feynman, Richard Feynman, self-driving car, side project, Silicon Valley, Skype, software as a service, speech recognition, statistical model, Steve Jobs, stochastic process, technology bubble, text mining, the scientific method, web application
My lucky break was being discovered by the co-founders of Endeca in 1999 and enlisted as chief scientist. My ten years there were an extraordinary adventure. Our initial ambition was to build a better way to find stuff on eBay. Like most startups, we pivoted, and we ultimately developed technology that revolutionized the search experience for online retail, as well as expanding into other domains like manufacturing, business intelligence, and government. After Endeca, I went to Google, where I worked on improving local search quality. Specifically, I led a team that matched the local search index against the web index to establish the official home pages of local businesses. It was a fun machine learning problem, and there was something very rewarding about improving the search experience on the world’s most popular web site.
I think that’s what’s going to raise the entire country’s standard of living and recreate the middle that’s been kind of squeezed out and pushed to the ends, which isn’t at all healthy for our country or society. The rise of data literacy will happen, and it will be tremendous for society. www.it-ebooks.info 149 CHAPTER 8 Claudia Perlich Dstillery Claudia Perlich is the Chief Scientist at Dstillery (formerly Media6Degrees) and teaches data mining for business intelligence in the MBA program of the Stern School of Business, New York University. In the top 50 of Forbes’ Top 100 Most Promising Companies in America, Dstillery is an advertising technology company that uses scientific methods to capture the full customer journey by collecting massive quantities of data from both the digital and the physical worlds to identify patterns that are unique to the brand and indicate consumer intent.
How does it compare to Bitly? Smith: It’s been really different, as they are two very different types of companies. At Bitly, the company dealt with a more latent data source. People use it and we see things happen. It was more about trying to capture people’s behaviors and understand what was going on in the Internet. At Rent the Runway, it’s a lot more of trying to support the business, so a lot of pure business intelligence and business analytics. The problem here is trying to figure out how we can put data into the product to drive business goals. Gutierrez: How do you explain what you do to someone not familiar with computer science, or physics, or data science? Smith: Like, how do I tell my mom what I’m doing? Well, my mom’s a bad choice since she loves computers. Okay, how about—how would I tell my sister?
Microchip: An Idea, Its Genesis, and the Revolution It Created by Jeffrey Zygmont
Albert Einstein, Bob Noyce, business intelligence, computer age, El Camino Real, invisible hand, popular electronics, side project, Silicon Valley, Silicon Valley startup, William Shockley: the traitorous eight
(The refund didn't go far enough: Busicom died in bankruptcy a few years later.) Yet even after it obtained ownership, the chip-maker haggled internally about what to do with the microprocessor. The rancor raged three months longer, until competitive cross-fertilization returned to settle the argument, traveling back from Dallas to Santa Clara. This time the stimulus wasn't a technical specification like the one Datapoint had spirited to TI. It was more generalized business intelligence, carried in the person of Ed Gelbach, who was hired away from Texas Instruments to serve as Intel's new marketing director in August 1971. The hire was a coup for Intel because TI was then the prime mover in semiconductors. Before Gelbach had left Dallas, TI had silently developed its own version of a programmable, single-chip computer. Aiming to power the Datapoint terminal, TI's Gary Boone designed and patented his own take on a microprocessor, finishing soon enough to demonstrate a working model in March 1971.
Recalling the secretive ways of Magnavox, Fosnough taped paper over the hall windows of the small lab while he built his prototype controller. He didn't want the idea to leak out into the wider appliance world. Essex engineers spent a lot of time calling on customers, kibitzing with kindred engineers who made washers, dryers, air conditioners, refrigerators, ranges, and microwaves. The visits kept them current so that Essex could create ever more effective relays and switches. But business intelligence seeped both ways, and Fosnough worried that if the rest of Essex R&D knew what he was up to, some loose-lipped 138 MICROCHIP staffer might say too much as he bent over plans and product drafts at an appliance company. Once an appliance engineer got wind of it, he'd mention it to every other control company he traded with. Fosnough reckoned that Essex competitors would be sure to copy the concept.
The Secret World of Oil by Ken Silverstein
business intelligence, clean water, corporate governance, corporate raider, Donald Trump, energy security, Exxon Valdez, failed state, Google Earth, offshore financial centre, oil shock, paper trading, rolodex, Ronald Reagan, WikiLeaks, Yom Kippur War
Nye also offered advice to Saif Gaddafi, the colonel’s son, on the dissertation he wrote for the London School of Economics. Gaddafi also counted on support from the Washington-based US-Libya Business Association, which was founded and funded by American oil companies. The group was headed by David Goldwyn, who worked at the Energy Department under Clinton and who, after retiring from government to run a consulting firm that provided “political and business intelligence” to industry, returned to public service as the State Department’s coordinator for international energy affairs during the Obama administration. A few others featured in this book include: • Former British prime minister Tony Blair, who netted about $150,000 for a twenty-minute speech in Azerbaijan, in which he said that President Ilham Aliyev was a leader with a “very positive and exciting vision for the future of the country.”
According to its federal tax filings, the association sought to “educate the public on the importance of US–Libya trade and investment, and facilitate the commercial and diplomatic dialogue between the two countries.” At least seven of its eight directors were registered lobbyists for oil companies. David Goldwyn—who had served at the Energy Department under Bill Clinton, and who then ran a consulting firm that provided “political and business intelligence” to oil companies—was hired to head the group. The USLBA spent over $1 million between 2006 and 2009, of which more than six hundred thousand dollars was used to pay Goldwyn’s firm. In February 2007, the USLBA brought together government officials from both countries at a gala dinner, held at the Charlie Palmer steakhouse, to honor a senior official of “The Great Socialist People’s Libyan Arab Jamahiriya.”
CTOs at Work by Scott Donaldson, Stanley Siegel, Gary Donaldson
Amazon Web Services, bioinformatics, business intelligence, business process, call centre, centre right, cloud computing, computer vision, connected car, crowdsourcing, data acquisition, distributed generation, domain-specific language, glass ceiling, pattern recognition, Pluto: dwarf planet, QR code, Richard Feynman, Richard Feynman, Ruby on Rails, shareholder value, Silicon Valley, Skype, smart grid, smart meter, software patent, thinkpad, web application, zero day, zero-sum game
Mosca: We are involved with the data dictionary, master data management, all of the attributes as well. Yeah, it's ultimately going to be a large business intelligence initiative. Siegel: Okay. So you're hired in this case to get down to the blade-of-grass level. Mosca: Absolutely. G. Donaldson: Dmitry, what do you see as some of the important issues confronting your industry today from a technology perspective? Cherches: Security and hacking are going to become more of a daily normal event. Unfortunately, many still believe that it's a lot easier to steal versus innovate. So, the CTO is responsible for making sure data is protected and mobile devices are not creating a threat. Being able to manage data and doing a lot of business intelligence from the data. That area is starting to become fairly standard. The technology is fairly affordable and if you don't have good KPI's (Key Performance Indicators), if you don't have good alerts that you can deliver to business stakeholders, you need to look into it ASAP.
Donaldson: As CTO, what do you see are transformative and emerging technologies that need to be tracked? Cherches: Definitely cloud, definitely virtualization. It's now part of the enterprise, and if you don't have it, you'd better do it now because it cuts costs and makes IT very flexible. Another one is the ability to handle large data. A lot of companies are going to be facing issues regarding enormous amounts of data, how to present it, and how to create business intelligence solutions out of it. There are not a lot of standards yet how to handle these petabytes of data, how to access it in an optimal, cost effective way. There are some open source applications to manage big data, and some very well-known large companies are starting to support those standards. S. Donaldson: How about cyber security? Cherches: Cyber security—well, that's a big subject. I always talk about one day walking into the office and everything is wiped out.
3D printing, additive manufacturing, Airbus A320, Albert Einstein, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, asset-backed security, augmented reality, barriers to entry, bitcoin, bounce rate, business intelligence, business process, business process outsourcing, call centre, capital controls, citizen journalism, Clayton Christensen, cloud computing, credit crunch, crowdsourcing, disintermediation, en.wikipedia.org, fixed income, George Gilder, Google Glasses, high net worth, I think there is a world market for maybe five computers, Infrastructure as a Service, invention of the printing press, Jeff Bezos, jimmy wales, London Interbank Offered Rate, M-Pesa, Mark Zuckerberg, mass affluent, Metcalfe’s law, microcredit, mobile money, more computing power than Apollo, Northern Rock, Occupy movement, optical character recognition, peer-to-peer, performance metric, Pingit, platform as a service, QR code, QWERTY keyboard, Ray Kurzweil, recommendation engine, RFID, risk tolerance, Robert Metcalfe, self-driving car, Skype, speech recognition, stem cell, telepresence, Tim Cook: Apple, transaction costs, underbanked, US Airways Flight 1549, web application
See also (http://www.businesswire.com/news/home/20111102005712/en/Phone-Bank, http://whatjapanthinks.com/2010/03/20/almost-two-thirds-use-net-banking-in-japan/) 15 For a definition of the Information Age, see http://en.wikipedia.org/wiki/Information_Age 16 A.H. Maslow, “A Theory of Human Motivation”, Psychological Review 50 (1943): 370–96. 17 Countrywide.com 18 Matt Coffin, “The next generation of mortgage lead generation”, LowerMyBills.com. Additional sources: Forrester Research Inc, Federal Trade Commission 19 “Online mortgage sites offer net gains”, Australasian Business Intelligence, 18 September 2006 20 Mortgagebot’s Benchmarks 2011 Report 21 Google Finance Australia 22 See http://en.wikipedia.org/wiki/Apple_II 23 The likes of Forbes have measured mass market adoption or critical mass by the benchmark of 25 per cent of the population for developed economies such as the United States, the United Kingdom, France, Germany, Australia, etc. or 100 million persons globally (See also http://photos1.blogger.com/blogger/4015/329/1600/technology_adoption_11.jpg).
As effectiveness has reduced, you increase the offer of the month to perhaps three or five different products that the staff member can choose from on the fly, depending on what he knows about the customer, but it is still hit and miss. You need to have intelligence built into the system in respect of the next best offer or the next best action for the customer. This requires the implementation of customer analytics, business intelligence and offering management solutions to take a range of offers each month, and match the right offer with the right client. It also requires each of the product teams to come up with a range of offers each month that can be triggered by key data points for specific customers. Most product teams are used to running just one acquisition-type campaign on a new product three or four times a year—and it takes a great deal of work with creative agencies and so forth to pull it off.
Don’t opt for a simple banner ad approach here as all you’ll do is upset your most valuable audience. You have to think about this and provide relevant offers. So don’t pitch a Gold credit card to someone who already has a Platinum card. Don’t pitch term deposits to a customer who is already a Premier account holder with a managed fund. Don’t pitch retirement plans to a student, etc. For this you need business intelligence and segmentation that create compelling, targeted offers that appeal. There are some other issues with selling behind the login, though—issues that have come as a result of marketers picking up some bad habits along the way. Combating banner blindness In a fairly brilliant piece of early usability testing, Jan Benway and David Lane of Rice University discovered in 1998 that users were starting to filter out “advertising banners”.
PostgreSQL 9 Admin Cookbook: Over 80 Recipes to Help You Run an Efficient PostgreSQL 9. 0 Database by Simon Riggs, Hannu Krosing
This activity is where you might reconsider current index choices. Many of these activities are mentioned in this chapter or throughout the rest of the cookbook. Some are not because they aren't so much technical tasks but more just planning and understanding of your environment. You might also find time to consider the following things: ff ff 240 Data quality: Are the contents of the database accurate and meaningful? Could the data be enhanced? Business intelligence: Is the data being used for everything that can bring value to the organization? 10 Performance & Concurrency In this chapter, we will cover the following: ff Finding slow SQL statements ff Collecting regular statistics from pg_stat* views ff Finding what makes SQL slow ff Reducing the number of rows returned ff Simplifying complex SQL ff Speeding up queries without rewriting them ff Why queries do not use an index ff How do force a query to use an index ff Using optimistic locking ff Reporting performance problems Introduction Performance and concurrency are two problems that are often tightly coupled—when concurrency grows, performance usually degrades, in some cases a lot.
There are usually two main reasons for wanting to do this, and often those reasons are combined: ff 300 High availability: Reducing the chances of data unavailability by having multiple systems each holding a full copy of the data. Chapter 12 ff Data movement: Allowing data to be used by additional applications or workloads on additional hardware. Examples are Reference Data Management, where a single central server might provide information to many other applications, and also Business Intelligence/Reporting Systems. Of course, both of those topics are complex areas, and there are many architectures and possibilities for doing each of those. What we will talk about here is data movement, where there is no transformation of the data—we simply copy the data from one PostgreSQL database server to another. So, we are specifically avoiding all discussion on ETL tools, EAI tools, inter-database migration, data warehousing strategies, and so on.
Bad Data Handbook by Q. Ethan McCallum
Amazon Mechanical Turk, asset allocation, barriers to entry, Benoit Mandelbrot, business intelligence, cellular automata, chief data officer, Chuck Templeton: OpenTable, cloud computing, cognitive dissonance, combinatorial explosion, commoditize, conceptual framework, database schema, en.wikipedia.org, Firefox, Flash crash, Gini coefficient, illegal immigration, iterative process, labor-force participation, loose coupling, natural language processing, Netflix Prize, quantitative trading / quantitative ﬁnance, recommendation engine, selection bias, sentiment analysis, statistical model, supply-chain management, survivorship bias, text mining, too big to fail, web application
These were the types of systems that I encountered as I started on my path to develop models for predicting homicides and shootings. Should I have been surprised by this? Absolutely not. What the government had done was something I had seen during the early days at OpenTable. We emphasized creating systems and focusing on the market rather than the technology; we raced ahead without giving the underlying architecture enough attention. In addition, business intelligence was still in its infancy. A good report would often just consist of output in a spreadsheet. At that point, data mining and predictive modeling were completely absent from the strategy. Then there was the perception component. Data that was of slightly questionable quality, or data that could possibly have contained any errors, could not be used. It was assumed if a small piece of data lacked veracity, all of the data needed to be disqualified.
We don’t know what we have, and we’ve never done any analysis before. We have an idea of what we have, but we’ve never done any analysis before. We know what we have, and we’ve tried answering specific questions, but we’re stuck. Case study: A Fortune 500 technology company had a process that generated data based on day-to-day operations. Historically, their data warehousing team performed most analytics tasks, such as traditional Business Intelligence (BI). This team’s primary responsibility was to develop relational database-driven tools, though they had also been dabbling in less-conventional analytics projects. They decided to bring in a data scientist to help with the latter endeavor. The belief was that the data scientist would magically find a golden nugget hidden in their data, which they could easily translate into some results.
Exponential Organizations: Why New Organizations Are Ten Times Better, Faster, and Cheaper Than Yours (And What to Do About It) by Salim Ismail, Yuri van Geest
23andMe, 3D printing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, bioinformatics, bitcoin, Black Swan, blockchain, Burning Man, business intelligence, business process, call centre, chief data officer, Chris Wanstrath, Clayton Christensen, clean water, cloud computing, cognitive bias, collaborative consumption, collaborative economy, commoditize, corporate social responsibility, cross-subsidies, crowdsourcing, cryptocurrency, dark matter, Dean Kamen, dematerialisation, discounted cash flows, distributed ledger, Edward Snowden, Elon Musk, en.wikipedia.org, ethereum blockchain, Galaxy Zoo, game design, Google Glasses, Google Hangouts, Google X / Alphabet X, gravity well, hiring and firing, Hyperloop, industrial robot, Innovator's Dilemma, intangible asset, Internet of things, Iridium satellite, Isaac Newton, Jeff Bezos, Kevin Kelly, Kickstarter, knowledge worker, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, lifelogging, loose coupling, loss aversion, Lyft, Marc Andreessen, Mark Zuckerberg, market design, means of production, minimum viable product, natural language processing, Netflix Prize, Network effects, new economy, Oculus Rift, offshore financial centre, p-value, PageRank, pattern recognition, Paul Graham, peer-to-peer, peer-to-peer model, Peter H. Diamandis: Planetary Resources, Peter Thiel, prediction markets, profit motive, publish or perish, Ray Kurzweil, recommendation engine, RFID, ride hailing / ride sharing, risk tolerance, Ronald Coase, Second Machine Age, self-driving car, sharing economy, Silicon Valley, skunkworks, Skype, smart contracts, Snapchat, social software, software is eating the world, speech recognition, stealth mode startup, Stephen Hawking, Steve Jobs, subscription business, supply-chain management, TaskRabbit, telepresence, telepresence robot, Tony Hsieh, transaction costs, Tyler Cowen: Great Stagnation, urban planning, WikiLeaks, winner-take-all economy, X Prize, Y Combinator, zero-sum game
Dashboards Extending the notion that decision-making in companies should be driven by data rather than by intuition, Dashboards offer an intuitive way to present complex information in a simple and cogent way. John Seely Brown and John Hagel have observed that although all of our large organizations are set up to scale efficiencies, in this new economy what we actually need to scale is learning. And while some very good business intelligence (BI) systems exist out there, they are set up largely to measure scaling of efficiency. What is needed now are new dashboards that measure the learning capability of organizations. And if those learning dashboards don’t emerge soon, big companies should consider requiring that their newly minted chief data officers (the hottest new C-Level position) build them. What, exactly, should learning Dashboards track?
Hence the need for the chief data officer, whose primary focus is managing data, finding the actionable information within, and then delivering it quickly, securely and in a useful form to every stakeholder in the organization. Key Opportunity Implications and Actions Externally driven IT Leverage external community (developers) and partnerships (startups, SaaS, companies) for new services/products and open platforms with open APIs (remix datasets, open source standards) and provide own metadata (access, remixing). Business intelligence (BI) Data management systems that use methodologies, processes, architectures and technologies to transform raw data into meaningful and useful business information (more effective strategic, tactical and operational insights and decision-making). A key heuristic: if you operate in a highly uncertain environment, make it simple (not too many variables); if you operate in a predictable environment, make it complex (use more variables to manage BI).
Digital Bank: Strategies for Launching or Becoming a Digital Bank by Chris Skinner
algorithmic trading, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, bank run, Basel III, bitcoin, business intelligence, business process, business process outsourcing, call centre, cashless society, clean water, cloud computing, corporate social responsibility, credit crunch, crowdsourcing, cryptocurrency, demand response, disintermediation, don't be evil, en.wikipedia.org, fault tolerance, fiat currency, financial innovation, Google Glasses, high net worth, informal economy, Infrastructure as a Service, Internet of things, Jeff Bezos, Kevin Kelly, Kickstarter, M-Pesa, margin call, mass affluent, mobile money, Mohammed Bouazizi, new economy, Northern Rock, Occupy movement, Pingit, platform as a service, Ponzi scheme, prediction markets, pre–internet, QR code, quantitative easing, ransomware, reserve currency, RFID, Satoshi Nakamoto, Silicon Valley, smart cities, software as a service, Steve Jobs, strong AI, Stuxnet, trade route, unbanked and underbanked, underbanked, upwardly mobile, We are the 99%, web application, Y2K
This means that the way in which you guard against data failings from external attack is by having the obvious data protections: firewalls, secure sign-on, dual authentication with triangulation of access, real-time business events monitoring and so on. What I mean by this is that banks should be moving towards much improved real-time tracking and business intelligence about their information flows, and this will alert them to any security breach. After all, most banks know that they will be breached. In fact, they know they cannot stop a breach. It will happen. The real question then is how you deal with it and how fast. That’s the key. This is why complex event monitoring of business intelligence flows with real-time alerts is a key focal point. The ability for a bank to keep its finger on the pulse of every transaction across its global operations will be the key to protecting against internal and external threats.
The Industries of the Future by Alec Ross
23andMe, 3D printing, Airbnb, algorithmic trading, AltaVista, Anne Wojcicki, autonomous vehicles, banking crisis, barriers to entry, Bernie Madoff, bioinformatics, bitcoin, blockchain, Brian Krebs, British Empire, business intelligence, call centre, carbon footprint, cloud computing, collaborative consumption, connected car, corporate governance, Credit Default Swap, cryptocurrency, David Brooks, disintermediation, Dissolution of the Soviet Union, distributed ledger, Edward Glaeser, Edward Snowden, en.wikipedia.org, Erik Brynjolfsson, fiat currency, future of work, global supply chain, Google X / Alphabet X, industrial robot, Internet of things, invention of the printing press, Jaron Lanier, Jeff Bezos, job automation, John Markoff, knowledge economy, knowledge worker, lifelogging, litecoin, M-Pesa, Marc Andreessen, Mark Zuckerberg, Mikhail Gorbachev, mobile money, money: store of value / unit of account / medium of exchange, new economy, offshore financial centre, open economy, Parag Khanna, peer-to-peer, peer-to-peer lending, personalized medicine, Peter Thiel, precision agriculture, pre–internet, RAND corporation, Ray Kurzweil, recommendation engine, ride hailing / ride sharing, Rubik’s Cube, Satoshi Nakamoto, selective serotonin reuptake inhibitor (SSRI), self-driving car, sharing economy, Silicon Valley, Silicon Valley startup, Skype, smart cities, social graph, software as a service, special economic zone, supply-chain management, supply-chain management software, technoutopianism, The Future of Employment, underbanked, Vernor Vinge, Watson beat the top human players on Jeopardy!, women in the workforce, Y Combinator, young professional
“Data: The Raw Material of the Information Age” and “The Geography of Future Markets” (chapters 5 and 6) examine both the expansiveness that big data will allow and the constraints that geopolitics will place on the global marketplace. Whereas land was the raw material of the agricultural age and iron was the raw material of the industrial age, data is the raw material of the information age. The Internet has become an ocean of jumbled, chaotic information, but now there is a way to connect this information and draw actionable business intelligence from it. Big data is transitioning from a tool primarily for targeted advertising to an instrument with profound applications for diverse corporate sectors and for addressing chronic social problems. At the same time, the industries of the future will both be created within the current geopolitical structure and transform it. In the 20th century, the dominant divide between political systems and markets was along the axis of left versus right.
We can now chew through enough data fast enough in a way that people can afford . . . and storage got cheap, so we can store lots of data . . . and then we can actually process it fast enough to make use of it.” Increases in data gathering and growth in computing power complement each other. The more data there is, the more investment there is in powerful computers and abundant storage to chew through the data and draw business intelligence from it. The more powerful computers are, the easier it is to gather large amounts of data and produce larger and more in-depth data sets. Big data is inherently contradictory. It is both intimate and expansive. It examines small facts and aggregates these finite facts into information that can be both comprehensive and personalized. Academics have likened it to both a microscope and telescope—a tool that allows us to both examine smaller details than could previously be observed and to see data at a larger scale, revealing correlations that were previously too distant for us to notice.
NumPy Cookbook by Ivan Idris
Rosario Acquisition Editor Usha Iyer Lead Technical Editor Ankita Shashi Technical Editors Merin Jose Rohit Rajgor Farhaan Shaikh Nitee Shetty Copy Editor Insiya Morbiwala Project Coordinator Vishal Bodwani Proofreader Clyde Jenkins Indexer Monica Ajmera Mehta Production Coordinators Arvindkumar Gupta Manu Joseph Cover Work Arvindkumar Gupta Manu Joseph About the Author Ivan Idris has an MSc in Experimental Physics. His graduation thesis had a strong emphasis on Applied Computer Science. After graduating, he worked for several companies as a Java Developer, Data Warehouse Developer, and QA Analyst. His main professional interests are business intelligence, big data, and cloud computing. He enjoys writing clean, testable code, and interesting technical articles. He is the author of NumPy 1.5 Beginner's Guide. You can find more information and a blog with a few NumPy examples at ivanidris.net. I would like to dedicate this book to my family and friends. I would like to take this opportunity to thank the reviewers and the team at Packt for making this book possible.
Blackwater: The Rise of the World's Most Powerful Mercenary Army by Jeremy Scahill
air freight, anti-communist, Berlin Wall, Bernie Sanders, business climate, business intelligence, centralized clearinghouse, collective bargaining, Columbine, facts on the ground, Fall of the Berlin Wall, Naomi Klein, private military company, Project for a New American Century, Robert Bork, Ronald Reagan, school choice, school vouchers, stem cell, urban planning, zero-sum game
Pizarro also sold his services to defense and weapons companies—in both the United States and in Europe—seeking to break into Latin American markets. He would tell these companies, “Well, let’s say you pay me $10,000 a month times three months, I will provide you with enough information and enough business intelligence so your sales-people will know exactly which doors to knock, to which officers they’re supposed to address, and how and when and for how much and for how long.” Pizarro said he made enough money selling “business intelligence” that he decided in early 2003 to “step away from the company and enjoy the money, enjoy my free time.” Leaving the day-to-day operations of Red Tactica to his business partners, Pizarro began writing for a German magazine focused on military technology. In February 2003, as the United States prepared to invade Iraq, a producer at CNN’s Spanish-language channel contacted Pizarro and asked him to come to the network’s Washington bureau to apply for a possible position with the network as a commentator on the war.
Eventually, Pizarro struck up a relationship with virtually every defense and military attaché from “friendly” Latin American nations and earned a reputation as a go-to guy for Latin American countries seeking to purchase specialized weapons systems from major defense companies. Pizarro hotly denied that he was an arms dealer and scoffed at the label. Instead, he said, he was selling “business intelligence” to Latin American officials he characterized as essentially paying him to do their jobs. “A military attaché by definition is a gift, is a reward, is a promotion, is a vacation in Washington. You’re not supposed to actually work,” Pizarro said. “That is in the Latino world. For us, if you’re a general and you get promoted to a senior general, you get a year of vacation, a paid vacation with your entire family in Washington, D.C.
CIOs at Work by Ed Yourdon
8-hour work day, Apple's 1984 Super Bowl advert, business intelligence, business process, call centre, cloud computing, crowdsourcing, distributed generation, Donald Knuth, Flash crash, Googley, Grace Hopper, Infrastructure as a Service, Innovator's Dilemma, inventory management, Julian Assange, knowledge worker, Mark Zuckerberg, Nicholas Carr, rolodex, shareholder value, Silicon Valley, six sigma, Skype, smart grid, smart meter, software as a service, Steve Ballmer, Steve Jobs, Steven Levy, the new new thing, the scientific method, WikiLeaks, Y2K, Zipcar
You can imagine, as an employee, if I have that information at my fingertips, I’m a lot better equipped to answer those questions. One of the things we’ve started doing is tweeting what the status is on our outages. And a certain portion of our customer base really likes getting information that way, so I think there’s a great opportunity in that mobility space, especially to engage our customers in more two-way communication. The second area where there’s a huge opportunity for us is on data analytics. Business intelligence is going to be huge because we have so many sensors out on our network now that we can better predict when an outage will occur. We’ll never be able to predict them all because you don’t know where lightning’s going to hit. You don’t know when somebody’s going to run into a pole in their car. But there are sensors that allow us to look at our equipment and better tell if you’ve got a transformer that’s getting ready to blow.
., 87 Arizona Public Service (APS) Company, 66, 211, 223 Arizona State University, 227 ARPANET, 19, 117, 135 Art of Computer Programming, 2 Atlanta-based Southern Company, 191 AT&T, 191, 249 B Ballmer, Steve, 39 Bank of Boston, 47 Baylor-Grapevine Board of Trustees, 47 Bedrock foundation, 249 Bell Atlantic Mobile, 231 Bell Labs, 2, 249 BlackBerry, 60, 96, 116, 121, 171, 184, 246, 261, 296, 317 Blalock, Becky, 182, 191, 215 adaptability, 192 Air Force brat, 191, 192 Atlanta-based Southern Company, 191 banking industry, 203 Boucher, Marie, 196 brainstorm, 202 24/7 business, 199 business intelligence, 204 cloud computing, 205 cognitive surplus, 206 cognitive time, 206 Coker, Dave, 196 communication and education, 200 Community and Economic Development, 194 consumer market, 202 cybersecurity, 207, 209 data analytics, 204, 205 disaster recovery, 209 distributed generation, 204 distribution organization, 201 Egypt revolution, 198 farming technology, 206 finance backgrounds/marketing, 200, 209 Franklin, Alan, 193 Georgia Power, 191 Georgia Power Management Council, 193 global society, 206 Google, 198 incredible technology, 195 Industrial Age, 206 Information Age, 206 InformationWeek's, 196 infrastructure, 202 intellectual property, 196 intelligence and redundancy, 207 Internet, 198, 206 leapfrog innovations, 205 mainframe system, 207 marketing and customer service, 193, 200 MBA, finance, 192 microfiche, 207 microwave tower, 207 mobile devices, 203 mobility and business analytics, 205 Moore's Law, 205 new generation digital natives, 197 flexible and adaptable, 199 innovation and creativity, 199 superficial fashion, 198 Olympic sponsor, 193 out pushing technology, 202 reinforcement, 201 sense of integrity, 200 Southern Company, 194, 198, 201, 207 teamwork survey, 201 technology lab, 202 undergraduate degree, marketing, 192 virtualization, 205 VRU, 203 Ward, Eileen, 196 wire business, 201 world-class customer service, 203 Bohlen, Ken, 211 American Production Inventory Control Society, 211 Apple, 217 APS, 211, 223 ASU, 227 benchmarking company, 216 chief innovation officer, 229 Citrix, 217 cloud computing, 218, 219 cognitive surplus, 220 DECnet, 212 Department of Defense, 222 distributed computing, 217 energy industry, 214 gizmo/whiz-bang show, 216 GoodLink, 217 hard-line manufacturing, 218 home computing, 219 home entertainment, 219 Honeywell, 219 HR generalists, 215 information technology department, 211 Intel machines, 217 John Deere, 213 just say yes program, 223 Lean Six Sigma improvement process, 211 Linux, 220 MBA program, 214 mentors, 213 national alerts, 224 North American universities, 228 paradigm shifts, 218, 220 PDP minicomputers, 212 Peopleware, 226 prefigurative culture, 221 R&D companies, 218 Rhode Island, 226 role models, 213 San Diego Fire Department, 224 security/privacy issues, 217 skip levels, 223 smart home concepts, 219 smartphone, 217 social media, 225 Stead, Jerry, 214 Stevie Award, 211 Storefront engineering, 212 traditional management, 219, 226 Twitter, 224 vocabulary, 221 Waterloo operations, 213 Web 2.0 companies, 227 Web infrastructure, 215 wikipedia, 220 Y2K, 222 Botnets, 23 Brian's and Rob Pike's, 2 Bristol-Myers Squibb, 33 Broadband networks, 241 Brown, 227 Bryant, 227 BT Global Services, 253 BT Innovate & Design (BTI&D), 253 Bumblebee tuna, 130 C Career writing technology, 67 CASE tools, 232 Cash, Jim, 50 Christensen, Clyde, 212 Chrome, 14, 18 Chrysler Corporation, 175 Citibank, 337 Citicorp, 313 Citrix, 217 Client-server-type applications, 59 Cloud computing, 218, 219, 239, 240, 261, 262, 310, 311, 313 Cloud technology, 62 CNN, 54 COBOL, 250 Cognitive surplus, 20, 79, 206, 291 College of Engineering, University of Miami, 113 Columbia University, 1 Community and Economic Development, 194 Computer Sciences Corporation, 35 Computerworld magazine, 196 Consumer-oriented technology, 22 Content management system, 133 Corporate information management (CIM) program, 309 Corporate Management Information Systems, 87 Corvus disk drive, 36 Customer Advisory Boards of Oracle, 191 Customer-relationship management (CRM), 56 Cutter Business Technology Council, 173 D Dallas Children's Medical Center Development Board, 48 DARPA, 19 DDoS attacks and security, 81 DECnet, 212 Dell Platinum Council, 113 DeMarco, Tom, 16, 226 Department of Defense, 222, 329, 332 Detroit Energy, 252 Digital books, 30 Digital Equipment, 48 Distributed computing, 217 Dodge, 189 Dogfooding, 11, 37, 38, 236 DTE Energy, 173 DuPont Dow Elastomers, 151 E Educational Testing Service (ETS), 151 E-government, 282, 285 Electrical distribution grid, 182 Elementary and Secondary Education Strategic Business Unit, 151 Elements of Programming Style, 2 Ellyn, Lynne, 173 advanced technology software planning, 175 Amazon, 184 artificial intelligence group, 175 Association for Women in Computing, 173 benchmark, 180, 181 BlackBerries, 184 Burns, Ursula, 175 Chrysler, 176 Cisco, 186 cloud computing, 183, 184 component-based architecture, 186 corporate communications customer service, 185 Crain's Detroit Business, 173 cyber security threats, 177 degree of competence, 187 diversity and sophistication, 182 DTE Energy, 173 energy trading, 176 engineering and science programs, 188 enterprise business systems policy, 186 executive MBA program, 176 Facebook, 185 fresh-out-of-the-university, 187 General Electric, 174 Google, 184 Grace Hopper, 174 grid re-automation, 182 Henry Ford Hospital, 174 internal social media, 185 International Coaching Federation, 178 iPads, 184 IP electrical grids, 182 iPod applications, 182 IT budgets, 186 IT responsibilities, 176 Java, 186 level of sophistication, 179 lobbying efforts, 181 medical computing, 175 Miller, Joan, 174 Mulcahey, Anne, 175 Netscape, 175 neuroscience leadership, 189 object-oriented programming, 186 Oracle, 186 peer-level people, 179 people system, 177 policies and strategies, 180 Radio Shack, 180 remote access capacity, 189 security tool and patch, 183 sense of community, 180 Shipley, Jim, 174 smart grid, 177, 182 smart meters, 182 smart phone applications, 183 swarming, 179 technical competence, 178, 179 Thomas, Marlo, 174 Twitter, 185 UNITE, 181 vendor community, 186 virtualization, 183, 184 Xerox, 175 E-mail, 9 Employee-relationship management (ERM), 56 Encyclopedia, 115 Encyclopedia Britannica, 292 ERP, 123 F Facebook, 244 Ellyn, Lynne, 185 Sridhara, Mittu, 73, 84 Temares, Lewis, 116, 121, 131 Wakeman, Dan, 169 Federal information technology investments, 299 Flex, 236 Ford, 102 Ford, Monte, 47 agile computing, 59 agile development, 62, 66 airplanes, 51 American Airlines, 47 Arizona Public Services, 66 Bank of Boston, 47 Baylor-Grapevine Board of Trustees, 47 BlackBerry, 60 board of Chubb, 51 board of Tandy, 51 business organizations, 63 business school, dean, 50 career writing technology, 67 client-server-type applications, 59 cloud technology, 62 CNN, 54 common-sense functionality, 49 consumer-based technology, 60 CRM, 56 Dallas Children's Medical Center Development Board, 48 Digital Equipment, 48 ERM, 56 financial expert, 69 frequent-flier program, 57 frontal lobotomy, 57 Harvard Business Review, 50 HR policies, 65 IBM, 48 information technology, 47, 52 Internet, 54 Internet-based protocol, 59 iPhone, 52 IT stuff, 58 Knight Ridder, 51 legacy apps, 59 mainframe-like applications, 59 management training program, 64 marketing and technical jobs, 48 Maynard, Massachusetts mill, 48 MBA program, 50 mentors, 49 Microsoft, 50 mobile computing, 62 New York Times, 53 operations center, 54 PDP-5, 49 PDP-6, 49 Radio Shack, 51 revenue management, 57 role models, 49 security paradigms, 62 self-service machine, 57 Silicon Valley companies, 68 smartphones, 54 social networking, 51, 53, 56, 58 stateful applications, 59 techie department, 48 The Associates First Capital Corporation, 47 transmission and distribution companies, 47 wireless network, 59 YouTube, 65 Fort Worth, 226 Free software foundation, 19 Fried, Benjamin, 1, 241 agile development, 25 agile methodologies, 26 Apple Genius Bar, 8 ARPANET, 19 Art of Computer Programming, 2 Bell Labs, 2 books and records, accuracy, 25 botnets, 23 Brian's and Rob Pike's, 2 cash-like principles, 29 CFO, 4 check writers, 18 chrome, 14, 18 classic computer science text, 1 cognitive surplus, 20 Columbia University, 1 compensation management, 7 competitive advantage, 9, 18 computer science degree, 1 computer scientists, 6 consumer-driven computing, 12 consumer-driven software-as-a-service offerings, 12 consumer-driven technology, 12 consumer-oriented technology, 14, 22 corporate leadership, 25 cost centers, 4 DARPA, 19 decision makers, 17 decision making, 13 360-degree performance management, 7 detroit energy, 30 digital books, 30 document workbench, 2 dogfooding, 11 e-books, 29 Elements of Programming Style, 2 e-mail, 9 end-user support, 7 engineering executive group, 4 European vendors, 6 file servers and print servers, 17 Folger Library editions, 30 free software foundation, 19 German company, 13 German engineering, 13 Gmail, 15 Godot, 26 Google, 1 books, 29 products, 5, 10 software engineers, 6 hiring managers, 6 HR processes and technologies, 6 IBM model, 13 instant messaging, 9 Internet age, 6 interviewers, training, 6 iPad, 29 iPhone, 29 IPO, 3 IT, engineering and computer science parts, 4 Knuth's books, 2 Linux machine, 8 Linux software, 19 machine running Windows, 8 Macintosh, 8 Mac OS, 9 macro factors, 11 Managing Director, 1 mentors, 1 microcomputers, 18 Microsoft, 5 Minds for Sale, 20 Morgan Stanley, 1–3, 5, 16 nonacademic UNIX license, 2 nontechnical skills, 5 oil exploration office, 17 open-source phone operating system, 20 outlook, 15 PARC, 19 performance review cycles, 7 personal computer equipment, 15 post-Sarbanes-Oxley world, 25 project manager, 13 quants, 24 rapid-release cycle, 26 R&D cycle, 24 regression testing, 27 role models, 1 shrink-wrapped software, 14 signature-based anti-virus, 22 smartphone, 20, 27 social contract, 8 society trails technology, 21 software engineering tool, 13 software installation, 14 supply chain and inventory and asset management, 10 SVP, 4 telephony, 17 ten things, 13 TMRC, 19 TROFF, 2 typesetter workbench, 2 UI designer, 14 university computing center, 28 videoconferencing, 12 Visicalc, 24 Wall Street, 23 Walmart, 6 waterfall approach, 25 XYZ widget company, 5 YouTube video, 20 G Gates, Bill, 39, 50 General Electric, 134 General Foods, 309, 326–328 General Motors, 33, 321, 329, 332 George Mason School of Information Technology, 309 Georgia Power Company, 191–193, 196 Georgia Power Management Council, 193 German company, 13 German engineering, 13 German manufacturing company, 232 Gizmo/whiz-bang show, 216 Gmail, 15 GoodLink, 217 Google, 1, 84, 85, 117, 217, 219, 220, 222, 235, 241, 263, 302, 319 apps, 314 books, 29 commercial products, 10 model, 293 Government Accountability Office (GAO), 305 4G program, 250 4G smartphone, 235 GTE, 231 Gupta, Ashish aspiration, organization, 256 bandwidth and network infrastructure, 267 BlackBerry, 261 business and customer outcomes, 274 capital investment forums, 269 career progression, 255 cloud-based shared infrastructure model, 263 cloud computing, 261, 262 collaboration, 272 communications infrastructure, 258 compute-utility-based model, 262 control and integrity, 268 core competency, 255 core network infrastructure, 267 core strengths, 256 cost per unit of bandwidth, 267 customer demands, 268 data protection, 261, 262 decision-making bodies, 269 demographics, 272, 273 device convergence, 263 dogfooding, 259 employee flexibility, 260, 264 engagement and governance, 269 enterprise market segment, 261 equipment management, 260 executive MBA, 256 fourth-generation LTE networks, 267 functional service departments, 270 Global Services, distributed organization, 257 Google, 263, 275 Google Apps, 266 handheld devices, 265 hastily formed networks, 258 IMF, 266 innovation and application development, 265 iPad, 257, 260, 261, 266,267 iPhone, 266 Japan, 257, 258 London Business School, 253 management functions, 257 management sales functions, 257 market segments, 259 MBA, General Management, 253 measurements, 271 messaging with voice capability, 264 mini-microcomputer model, 261 mobile communications network, 258 mobile-enabling voice, 259 mobile phone network, 260 mobile traffic explosion, 265 network infrastructures, 265 network IT services, 254 network quality, 257 new generation digital natives, 271 disadvantages, 273 Google, 273 opportunities, 273 Olympics, 263 opportunities, 275 organizational construct, 272 outsourced network IT services, 259 outsourcing, 271 per-use-based model, 262 portfolio and business alignment, 274 Portfolio & Service Design (P&SD), 253 primary marketing thrust, 264 product development thrust, 264 product management team, 259 project and program management, 255 resource balance, 270 scalability, 262 security, 262 Selley, Clive, 254, 255 service delivery organization, 254 single-device model, 264 smart devices, 267 smart phones, 266 telecommunications capability, 259 upward-based apps, 264 virtualization, 261 voice-over-IP connections, 258 Windows platform, 261 Gurnani, Roger, 231 accounting/finance department, 233 analog cellular networks, 250 AT&T, 249 bedrock foundation, 249 Bell Atlantic Mobile, 231 Bell Labs, 249 blogs, 244 broadband networks, 241 business benefits, 237 business device, 240 business executives, 238 business leaders, 248, 249 business relationship management, 248 buzzword, 239 CASE tools, 232 cloud computing, 239, 240 COBOL, 250 consumer and business products, 231 consumer electronics devices, 241 consumer telecom business, 233 customer-engagement channel, 244 customer forums, 244 customer support operations, 251 customer-touching channels, 236 degree of control, 246 distribution channel, 250 dogfooding, 236 ecosystem, 243, 249 enterprise business, 233 ERP systems, 236 face-to-face communications, 244 FiOS product, 235 flex, 236 "follow the sun" model, 239 German manufacturing company, 232 4G program, 250 4G smartphone, 235 hardware/software vendors, 247 information assets, 245 information technology strategy, 231 intellectual property rights, 244 Internet, 235, 239 iPhone, 243 Ivan, 232 Lowell, 232 LTE technology-based smartphone, 235 marketing, 251 MIT, 246 mobile technology, 234 Moore's law, 242 MP3 file, 235 network-based services, 240 Nynex Mobile, 233 P&L responsibility, 251 PDA, 238 personal computing, 235 product development, 234, 251 role models, 232 sales channels, 251 smartphones, 238 state-level regulatory issues, 251 state-of-the-art networks, 243 telecom career, 232 telephone company, Phoenix, 234 Verizon Communication, 231, 232 virtual corporations, 241 Web 2.0, 244 Williams Companies, 232, 233 WillTell, 233 wireless business, 233 H Hackers, 19 Harmon, Jay, 213 Harvard Business Review, 50 Harvard Business School, 331 Heller, Martha, 171 Henry Ford Hospital, 174 Hewlett-Packard piece, 129 Home computing, 219 Honda, 102 Honeywell, 219 Houghton Mifflin, 134, 136 I IBM, 48, 250 manpower, 311 model, 13 Indian IT outsourcing company, 255 Information technology, 52 Intel machines, 217 International Coaching Federation, 178 Internet, 9, 44, 54, 117, 235, 239, 316, 322 Internet-based protocol, 59 Interoperability, 341 iPads, 2, 94, 97, 184, 257, 260, 264, 267, 288, 289, 295, 296 IP electrical grids, 182 iPhones, 43, 52, 96, 101, 170, 181, 260, 264,296 iPod, 101 IT lifecycle management process, 37 Ivan, 232 J John Deere, 213 K Kansas, 226 Kernigan, Brian, 2 Knight Ridder, 51 Knuth, Donald, 2, 29 Kraft Foods Inc, 309 Krist, Nicholas, 28 Kundra, Vivek Clever Commute, 305 cognitive surplus, 303 command and control systems, 301 consumerization, 302 consumption-based model, 300 cyber-warfare, 301 Darwinian pressure, 302 desktop core configuration, 306 digital-borne content, 301 digital oil, 300, 307 digital public square, 304 enterprise software, 303 entrepreneurial startup model, 306 frugal engineering, 306 Google, 302 government business, 302 innovator's dilemma, 307 iPad, 302 IT dashboard, 302 leapfrog technology, 306 massive consumerization, 301 megatrends, 301 parameter security, 302 Patent Office, 305 pharmaceutical industry, 304 phishing attacks, 301 policy and strategic planning, 299 security and privacy, 301 server utilization, 300 social media and technology, 300, 306 storage utilization, 300 Trademark Office, 305 Wikipedia, 303 L LAN, 259 Lean Six Sigma improvement process, 211 Levy, Steven (Hackers), 19 Linux, 220 machine, 8 open-source software, 19 Lister, Tim, 226 London Business School, 73, 253, 256 Long-term evolution (LTE), 235 Lowell, 232 M MacArthur's intelligence officer, 327 Macintosh, 8 Mainframe computers, 118 Mainframe-like applications, 59 Marriott's Great America, 35 McDade, 327 McGraw-Hill Education, 133, 147, 150 Mead, Margaret, 221 Mendel, 311 Microcomputers, 18 Microsoft Corporation, 5, 11, 33, 36, 38, 41, 44, 46, 50, 156, 217, 223, 236, 250, 293 Microsoft Higher Education Advisory Group, 113 Microsoft's operational enterprise risk management, 33 Middlesex University, 189 Miller, Joan Apple products, 295 authority and accuracy, 292 award-winning ICT programs and services, 277 back locked-down information, 289 big-scale text issues, 294 big-time computing, 279 BlackBerry, 296 business management training, 281 business skills, 281 central government, 283 cognitive surplus, 291 community care project, 278 community development programs, 277, 278 computers, constituency office, 294 confidential information, 284 data management, 281 decision making, 286 democratic process, 288 economics degree, 278 e-government, 282, 285 electronic communication, 289 electronic-enabled public voice, 286 electronic information, 288 electronic media, 286 electronic records, 280, 284 electronic services, 294 e-mail, 289, 290, 295 forgiving technology, 296 front-office service, 282, 283 Google, 292 Google's cloud service, 290 Government 2, 287 Health and Social Care, 284 House business, 294 House of Lords, 288 ICT strategy, 289, 290 information management, 278 insurance company, 278 Internet information, 285 iPad, 288, 289, 296 IT data management, 279 management principle, 280 local government, 283 mainframe environment, 289 member-led activity, 287 messages, 289 Microsoft, 293 Microsoft's cloud service, 290 mobile electronic information, 284 mobile technology, 289 national organization, 284 network perimeters, 290 official government information, 285 on-the-job training, 281 organizational planning, 278 Parliamentary ICT, 277 project management, 279 public sector, 282 public transportation, 285 quango-type organizations, 283 representational democracy, 286 security, 290, 291 social care organization, 279 social care services, Essex, 278 social care systems, 284 social networking, 285 sovereignty, 291 sustainability and growth, 293 technical language, 294 technology skills, 281 transactional services, 285 transferability, 291 Web-based services, 285 Wikipedia, 291, 292 X-factor, 286 Minds for Sale, 20 Mitchell & Co, 333 MIT Media Labs, 149 Mobile computing, 62 Mobile technology, 234 Mooney, Mark, 133 artificial intelligence, 134 back-office legacy, 136 balancing standpoint, 145 BBC, 140 Bermuda Triangle, 135 BlackBerry shop, 142 Bureau of National Standards, 136 business model, 140 career spectrum, 144 cloud computing, 148 competitive intelligence and knowledge, 143 Connect, 141 customer-facing and product development, 135 customer-facing product space, 137 customer space and product development, 136 digital products development, 144 digital space and product, 146 educational and reference content, 139 educational products, 141 entrepreneur, 150 General Electric, 134 GradeGuru, 140 handheld devices, 142 hard-core technical standpoint, 146 hardware servers, 142 Houghton Mifflin, 134, 136 HTML, 138 industrial-strength product, 141 intellectual content, 148 Internet, 148 iPad, 138, 139, 142 iPhone, 142, 143 iTunes, 138 Klein, Joel, 147 learning management systems, 137 long-term production system, 141 Marine Corps, 134 McGraw-Hill Education, 133, 147 media development, 144 media space, 138, 142 mobile computing, 139 MOUSE, 150 online technology, 138 open-source capabilities, 142 Oracle quota-management system, 143 people's roles and responsibilities, 137 Phoenix, 149 product development, 149 publishing companies, 142 publishing systems, 137 Reed Elsevier, 133, 136 Salesforce.com, 144, 149 scalability testing, 145 senior business leaders, 146 social network, 148 soft discipline guidelines, 141 solar energy, 149 Strassmann, Paul, 135 technical skill set, 143, 144 testing systems integration, 145 The Shallows, 139 transactional systems, 142 trust and integrity, 145 TTS, QuickPro, and ACL, 144 Vivendi Universal, 134 War and Peace today, 139 Moore's law, 242 Morgan Stanley, 2, 3, 16 N NASA, 309, 333, 334 National Institute of Standards and Technology (NIST), 173 Naval Postgraduate School, 134 Netscape, 175 New Brunswick model, 282 News Corp., 147 New York Stock Exchange (NYSE), 87, 116, 223, 278 New York Times, 53 North American universities, 228 NSA/CIA software, 134 Nynex Mobile, 233 O Oil exploration office, 17 Open-source phone operating system, 20 Outlook, 15 P Pacer Software, 135 Paradigm shifts, 218, 220 Parks and Recreation Department, 126 PDP minicomputers, 212 Peopleware, 226 Personal computing, 235 Personal digital assistant (PDA), 238 Petri dish, 44 Phoenix, 211 Plauger, Bill, 2 Q Quants, 24 R Radio Shack, 51 Reed Elsevier, 133, 136 Reed, John, 335 Rubinow, Steve, 87 AdKnowledge, Inc., 87 agile development, 110 Agile Manifesto, 110 Archipelago Holdings Inc., 87 attributes, 108 capital market community, 91 cash/actual trading business, 88 channel marketing departments, 92 cloud computing, 97 CNBC, 89 collaborative technology, 95 collective intelligence, 95 communication skills, 102, 106 conference organizations, 99 consumer marketplace, 94 data center, 90 decision making, 105, 108 economy standpoint, 100 e-mail, 100 Fidelity Investments, 105 financial services, 92 IEEE, 101 innovative impression, 94 Internet, 98 iPad, 97 iPod device, 91 labor laws, 110 listening skills, 106 logical progression, 104 Mac, 96 mainframe, 104 management and leadership, 104, 105 market data system, 89 micro-second response time, 89 mobile applications, 94 multidisciplinary approach, 103 multimedia, 97 multi-national projects, 110 multiprocessing options, 99 network operating system, 103 NYSE Euronext, 87 open outside system, 88 parallel programming models, 99 personal satisfaction, 109 PR function, 106 proclaimed workaholic, 109 real estate business, 88 regulatory and security standpoint, 96 Rolodex, 94 Rubin, Howard, 99 server department, 97 software development, 89 sophisticated technology, 101 technology business, 88 technology integration, 91 trading engines, 90 typewriter ribbon, 94 virtualization, 98 Windows 7, 96 younger generation video games, 93 visual interfaces, 93 Rumsfeld, Donald, 222 S San Diego Fire Department, 224 Santa Clara University, 36 SAS programs, 131 Scott, Tony, 10, 33, 236 Android, 43 Apple Computer, 35 architectural flaw, 44 BASIC and Pascal, 35 Bristol-Myers Squibb, 33 Bunch, Rick (role model), 34 business groups, 42 COO, 39 Corporate Vice President, 33 Corvus disk drive, 36 CSC, 35 Defense department, 45 dogfooding, 37, 38 games and arcades, 35 General Motors, 33 IBM's role, 37 information systems management, 36 integrity factor, 40 Internet, 44 iPhone, 43 IT lifecycle management process, 37 leadership capability, 40 leisure studies, 34 macro-architectural threats, 44 Marriott's Great America, 35 math models, 36 Microsoft Corporation, 33, 36, 38, 41, 44, 46 Microsoft's operational enterprise risk management, 33 parks and recreation, 34 Petri dish, 44 playground leader, 42 product groups, 42 quality and business excellence team, 33 Santa Clara University, 36 Senior Vice President, 33 smartphone, 43 social computing, 38 Sun Microsystems, 36 theme park industry, 35 University of Illinois, 34 University of San Francisco, 36 value-added business, 33 Walt Disney Company, 33 Senior Leadership Technology and Product Marketing, 71 Shakespeare, 30 Shirky, Clay, 220 Sierra Ventures, 191 Silicon Valley companies, 68 Silicon Valley software factories, 323 Skype, 118 Smart Grid Advisory Committee, 177 Smartphones, 20, 27, 43, 54, 217, 238 Social care computer electronic record system, 279 Social computing, 38, 320 Social networking, 51, 53, 56, 58 Society trails technology, 21 SPSS programs, 131 Sridhara, Mittu, 71 Amazon, 76 American Airlines, 72 back-end computation and presentation, 80 banking, 77 B2B and B2C, 85 business/product departments, 82 business work context, 74 buzzword, 77 career aspiration, 73 career spans, 73 coders, 72 cognitive surplus, 79 competitive differentiation, 74 computing power, 78 contribution and energy, 85 convergence, 75 CPU cycles, 78 cross-channel digital business, 71 cultural and geographic implementation, 72 customer experience, 84, 85 customer profile, 76 data visualization, 79, 80 DDoS protection, 81 economies of scale, 77 elements of technology, 72 encryption, 82 end customer, 83 entertainment, 75 ERP system, 72 Facebook, 84 finance and accounting, 73 foster innovation and open culture, 81 friends/mentors/role models, 74 FSA, 76 gambling acts, 81 games, 79 gaming machines, 80 GDS, 72 global organization, 71 Google, 75, 84, 85 Group CIO, Ladbrokes PLC, 71 industry-standard technologies, 77 integrity and competence, 83 IT, 74, 82 KickOff app, 71 land-based casinos, 79 live streaming, 78 London Business School, 73 mobile computing, 78 multimedia, 84 new generation, 84 on-the-job training, 73 open-source computing, 79 opportunity, 80, 83 PCA-compliant, 81 personalization, 76 real-time systems, 74 re-evaluation, 81 reliability and availability, 77 security threats, 80 smart mobile device, 75 technology-intense customer, 85 top-line revenue, 74 trader apps, 82 true context, 73 underpinning business process, 76 virtualization, 78 Visa/MasterCard transactions, 78 Web 3.0 business, 76 web-emerging web channel, 76 Wikipedia, 79, 85 Word documents and e-mail, 82 work-life balance, 84 young body with high miles, 72 Zuckerberg, Mark, 73 Stead, Jerry, 214 Storefront engineering, 212 Strassmann, Paul, 228, 309 agile development, 340 Amazon EC2, 314 America information processors, 322 Annapolis, 340 AT&T, 332 backstabbing culture, 339 BlackBerry, 317 block houses, 319 CFO/CEO position, 337 CIM program, 309 Citibank, 337 Citicorp, 313, 339 cloud computing, 310, 311, 313 coding infrastructure, 341 communication infrastructure, 341 corporate information management, 329 Corporate Information Officer, 309 counterintelligence, 320 cyber-operations, 338 Dell server, 314 Department of Defense, 329, 332 Director of Defense Information, 309 employee-owned technology, 316 enterprise architecture, 316 exfiltration, 313 financial organizations, 320 firewalls and antiviruses, 312 General Foods, 309, 326–328 General Motors, 321, 329, 332 George Mason School of Information Technology, 309 Google apps, 314 government-supported activities, 326 Harvard Business School, 331 HR-related issues, 331 IBM manpower, 311 infiltration, 313 Internet, 316, 322 interoperability, 315, 317, 341 Kraft Foods Inc, 309 MacArthur's intelligence officer, 327 Machiavellian view, 327 mash-up, 316 military service, 331 NASA, 309, 333, 334 police department, economics, 312 powerpoint slides, 324 Radio Shack, 319 senior executive position, 334 service-oriented architecture, 316 Silicon Valley software factories, 323 social computing, 320 Strassmann's concentration camp, 318 structured methodologies, 342 U.S.
I'm Feeling Lucky: The Confessions of Google Employee Number 59 by Douglas Edwards
Albert Einstein, AltaVista, Any sufficiently advanced technology is indistinguishable from magic, barriers to entry, book scanning, Build a better mousetrap, Burning Man, business intelligence, call centre, commoditize, crowdsourcing, don't be evil, Elon Musk, fault tolerance, Googley, gravity well, invisible hand, Jeff Bezos, job-hopping, John Markoff, Marc Andreessen, Menlo Park, microcredit, music of the spheres, Network effects, P = NP, PageRank, performance metric, pets.com, Ralph Nader, risk tolerance, second-price auction, side project, Silicon Valley, Silicon Valley startup, slashdot, stem cell, Superbowl ad, Y2K
My respect for our two capricious, obstinate, provocative, and occasionally juvenile founders increased tenfold that day. There were other glimpses of Larry's thinking. He and Eric shared a list of possible strategies that included Google as the publisher of all content, where users would pay us and we would reimburse the creators of everything from books to movies to music. Google as a provider of market research and business intelligence based on what we knew about the world. Google as an infrastructure platform and communications provider tying email and web data together. Google as the leader in machine intelligence backed by all the world's data and massive computing power that learned as it went along. He had no small plans. Eric, on the other hand, was the voice of corporate pragmatism. These grand schemes would have to be paid for somehow.
John Barabino, who would head the syndication effort once the AOL deal was completed, became part of the team the day he was hired in February 2002. Alan understood exactly what was at stake, and he knew the opposition. Overture had been born as GoTo in Pasadena, right down the street from the Jet Propulsion Laboratory. Even though they drew talent from Cal Tech and were plenty smart, they weren't Silicon Valley smart. They weren't really a tech company, and they didn't have the business intelligence Alan saw at Google. On the other hand, they had come up with the idea of marrying ads with search and implemented it before Google had. When Omid called the AOL business-development unit run by David Colburn, AOL agreed to start talks with Google. Not because they intended to give us their business, but because they could use Google as a bludgeon to beat a better deal out of Overture when their contract came up for renewal.
Albert Einstein, barriers to entry, business intelligence, business process, cloud computing, Everything should be made as simple as possible, Hans Rosling, Richard Feynman, Richard Feynman, Silicon Valley, women in the workforce, Yogi Berra
The numbers in this simple bar chart are easy to read quickly, but the numbers become harder to see when a picture graph is used to show the same information. (Images in slides from iStockphoto.com.) You can always recognize truth by its beauty and simplicity. —Richard Feynman, physicist Stephen Few’s Graph Design IQ Test Stephen Few is one of the leading authorities in the field of data visualization and business intelligence. Through his company, Perceptual Edge, he focuses on the effective analysis and presentation of quantitative business information. Stephen is a remarkable presenter and a highly sought after speaker, trainer, and consultant. He is the author of several books on data information visualization, including his latest best-seller Now You See It: Simple Visualization Techniques for Quantitative Analysis (Analytics Press, 2009).
97 Things Every Programmer Should Know by Kevlin Henney
A Pattern Language, active measures, business intelligence, commoditize, continuous integration, crowdsourcing, database schema, deliberate practice, domain-specific language, don't repeat yourself, Donald Knuth, fixed income, general-purpose programming language, Grace Hopper, index card, inventory management, job satisfaction, loose coupling, Silicon Valley, sorting algorithm, The Wisdom of Crowds
He first worked as a programmer in 1980 writing applications for estate agents and solicitors in compiled BASIC on an Apple IIe. Upon graduating from Edinburgh University in 1987, he worked on the REKURSIV project before becoming a freelance contractor. Today, his primary software interests are agile practices and the resuscitation of legacy code. Chapter 1 Steve Berczuk Steve Berczuk is a software engineer at Humedica, where he develops business intelligence solutions for the healthcare industry. He has been developing software applications for over 20 years, and is the author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration (Addison-Wesley Professional). In addition to developing software, he enjoys helping teams deliver more effectively through the use of agile methods and software configuration management.
Python Network Programming Cookbook by M. Omar Faruque Sarker
I would also like to thank my Google Summer of Code mentor, Patirica Tressel. Tom Stephens has worked in software development for nearly 10 years and is currently working in embedded development dealing with smartcards, cryptography, and RFID in the Denver metro area. His diverse background includes experience ranging from embedded virtual machines to web UX/UI design to enterprise Business Intelligence. He is most passionate about good software design, including intelligent testing and constantly evolving practices to produce a better product with minimal effort. Deepak Thukral is a polyglot who is also a contributor to various open source Python projects. He moved from India to Europe where he worked for various companies helping them scale their platforms with Python. www.PacktPub.com Support files, eBooks, discount offers, and more You might want to visit www.PacktPub.com for support files and downloads related to your book.
Sex, Lies, and Pharmaceuticals: How Drug Companies Plan to Profit From Female Sexual Dysfunction by Ray Moynihan, Barbara Mintzes
A key aim was to win widespread acceptance of the idea that a woman’s common sexual difficulties might be the sign of a treatable dysfunction. For many researchers, all this activity was bringing what they regarded as long-overdue recognition to women’s sexual suffering, and legitimacy to its study. For the drug companies, it was a strategic part of the planning for what was being billed as the next billion-dollar market. A forward-looking business intelligence report in 2003 named FSD drugs as an area of great future growth for the pharmaceutical industry, part of the burgeoning ‘lifestyle’ market including medicines for baldness, smoking cessation and obesity.38 The report was prepared for industry insiders and, with a hefty price tag, was never intended for public consumption. However, a leaked copy described how drug companies were ‘expanding the patient pool’ by using marketing campaigns to change public perceptions about things that used to be considered part of normal life.
Airbnb, business intelligence, cloud computing, financial independence, Google Glasses, hiring and firing, Isaac Newton, Jeff Bezos, Marc Andreessen, Mark Zuckerberg, move fast and break things, move fast and break things, new economy, nuclear winter, Peter Thiel, Productivity paradox, random walk, Ronald Reagan, Silicon Valley, six sigma, Steve Ballmer, Steve Jobs
FINAL THOUGHT You might think that so much time spent on promotions and titles places too much importance and focus on silly formalisms. The opposite is true. Without a well thought out, disciplined process for titles and promotions, your employees will become obsessed with the resulting inequities. If you structure things properly, nobody other than you will spend much time thinking about titles other than Employee of the Month. WHEN SMART PEOPLE ARE BAD EMPLOYEES In business, intelligence is always a critical element in any employee, because what we do is difficult and complex and the competitors are filled with extremely smart people. However, intelligence is not the only important quality. Being effective in a company also means working hard, being reliable, and being an excellent member of the team. When I was a CEO, this was one of the most difficult lessons for me to learn.
Big Data: A Revolution That Will Transform How We Live, Work, and Think by Viktor Mayer-Schonberger, Kenneth Cukier
23andMe, Affordable Care Act / Obamacare, airport security, AltaVista, barriers to entry, Berlin Wall, big data - Walmart - Pop Tarts, Black Swan, book scanning, business intelligence, business process, call centre, cloud computing, computer age, correlation does not imply causation, dark matter, double entry bookkeeping, Eratosthenes, Erik Brynjolfsson, game design, IBM and the Holocaust, index card, informal economy, intangible asset, Internet of things, invention of the printing press, Jeff Bezos, lifelogging, Louis Pasteur, Mark Zuckerberg, Menlo Park, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, obamacare, optical character recognition, PageRank, performance metric, Peter Thiel, Post-materialism, post-materialism, random walk, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, smart grid, smart meter, social graph, speech recognition, Steve Jobs, Steven Levy, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, Turing test, Watson beat the top human players on Jeopardy!
It’s not that the system doesn’t know the actual total; it’s that as the scale increases, showing the exact figure is less important. Besides, the amounts may be changing so quickly that a specific figure would be out of date the moment it appeared. Similarly, Google’s Gmail presents the time of recent messages with exactness, such as “11 minutes ago,” but treats longer durations with a nonchalant “2 hours ago,” as do Facebook and some others. The industry of business intelligence and analytics software was long built on promising clients “a single version of the truth”—the popular buzz words from the 2000s from the technology vendors in these fields. Executives used the phrase without irony. Some still do. By this, they mean that everyone accessing a company’s information-technology systems can tap into the same data; that the marketing team and the sales team don’t have to fight over who has the correct customer or sales numbers before the meeting even begins.
Affordable Care Act / Obamacare, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Donald Trump, en.wikipedia.org, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta analysis, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, publication bias, QR code, randomized controlled trial, risk-adjusted returns, Ronald Reagan, selection bias, statistical model, The Signal and the Noise by Nate Silver, Thomas Bayes, Tim Cook: Apple, wikimedia commons, Yogi Berra
In his litigation work, he guides companies and outside counsel on the appropriate use and interpretation of complex data sets, and has served as an expert witness in some of today’s most high-stakes corporate lawsuits. On the business analytics side, Dr. Johnson helps companies translate their complex internal data sets into strategic, actionable information across a variety of business settings including human resources, finance, marketing, manufacturing, and business intelligence. Both aspects share the need to understand—and properly apply—large, complex sets of data. He applies this same skill to his writing and speaking, where he helps audiences avoid common pitfalls people make when confronted with data, so they can become more confident and discerning consumers of data and make better decisions in their professional and personal lives. Dr. Johnson is a frequent presenter on economic topics and the use of data, and has also authored numerous papers across his areas of expertise. 221158 i-xiv 1-210 r4ga.indd 205 2/8/16 5:58:50 PM 206 About the Authors Dr.
Martin Kleppmann-Designing Data-Intensive Applications. The Big Ideas Behind Reliable, Scalable and Maintainable Systems-O’Reilly (2017) by Unknown
active measures, Amazon Web Services, bitcoin, blockchain, business intelligence, business process, c2.com, cloud computing, collaborative editing, commoditize, conceptual framework, cryptocurrency, database schema, DevOps, distributed ledger, Donald Knuth, Edward Snowden, ethereum blockchain, fault tolerance, finite state, Flash crash, full text search, general-purpose programming language, informal economy, information retrieval, Internet of things, iterative process, John von Neumann, loose coupling, Marc Andreessen, natural language processing, Network effects, packet switching, peer-to-peer, performance metric, place-making, premature optimization, recommendation engine, Richard Feynman, Richard Feynman, self-driving car, semantic web, Shoshana Zuboff, social graph, social web, software as a service, software is eating the world, sorting algorithm, source of truth, SPARQL, speech recognition, statistical model, web application, WebSocket, wikimedia commons
For example, if your data is a table of sales transactions, then analytic queries might be: • What was the total revenue of each of our stores in January? • How many more bananas than usual did we sell during our latest promotion? • Which brand of baby food is most often purchased together with brand X diapers? 90 | Chapter 3: Storage and Retrieval These queries are often written by business analysts, and feed into reports that help the management of a company make better decisions (business intelligence). In order to differentiate this pattern of using databases from transaction processing, it has been called online analytic processing (OLAP) .iv The difference between OLTP and OLAP is not always clear-cut, but some typical characteristics are listed in Table 3-1. Table 3-1. Comparing characteristics of transaction processing versus analytic systems Property Main read pattern Transaction processing systems (OLTP) Small number of records per query, fetched by key Analytic systems (OLAP) Aggregate over large number of records Main write pattern Random-access, low-latency writes from user input Bulk import (ETL) or event stream Primarily used by End user/customer, via web application Internal analyst, for decision support What data represents Latest state of data (current point in time) History of events that happened over time Dataset size Gigabytes to terabytes Terabytes to petabytes At first, the same databases were used for both transaction processing and analytic queries.
At the same time, by having the extensibility of being able to run arbi‐ trary code and read data in arbitrary formats, they retain their flexibility advantage. Specialization for different domains While the extensibility of being able to run arbitrary code is useful, there are also many common cases where standard processing patterns keep reoccurring, and so it is worth having reusable implementations of the common building blocks. Tradition‐ ally, MPP databases have served the needs of business intelligence analysts and busi‐ ness reporting, but that is just one among many domains in which batch processing is used. Another domain of increasing importance is statistical and numerical algorithms, which are needed for machine learning applications such as classification and recom‐ mendation systems. Reusable implementations are emerging: for example, Mahout implements various algorithms for machine learning on top of MapReduce, Spark, and Flink, while MADlib implements similar functionality inside a relational MPP database (Apache HAWQ) . 428 | Chapter 10: Batch Processing Also useful are spatial algorithms such as k-nearest neighbors , which searches for items that are close to a given item in some multi-dimensional space—a kind of simi‐ larity search.
Top Secret America: The Rise of the New American Security State by Dana Priest, William M. Arkin
airport security, business intelligence, dark matter, drone strike, friendly fire, Google Earth, hiring and firing, illegal immigration, immigration reform, index card, Julian Assange, profit motive, RAND corporation, Ronald Reagan, WikiLeaks
Commercial real-estate agent Lane, the building’s owner, had his engineers reinforce the steel beams to meet government specifications for security. The senior vice president of a local real estate firm has become something of a snoop himself when it comes to his NSA neighborhood. At fifty-five, he has lived and worked in its shadow all his life and has schooled himself on its growing presence in his community. He collects business intelligence. He has his own network of informants, executives like himself hoping to make a killing off an organization many of his neighbors don’t know a thing about. Lane takes note when the NSA or another secretive government organization leases another building, hires more contractors, and expands its outreach to the local business community. He’s been following construction projects, job migrations, corporate moves.
How to Fix Copyright by William Patry
A Declaration of the Independence of Cyberspace, barriers to entry, big-box store, borderless world, business intelligence, citizen journalism, cloud computing, commoditize, creative destruction, crowdsourcing, death of newspapers, en.wikipedia.org, facts on the ground, Frederick Winslow Taylor, George Akerlof, Gordon Gekko, haute cuisine, informal economy, invisible hand, Joseph Schumpeter, Kickstarter, knowledge economy, lone genius, means of production, moral panic, new economy, road to serfdom, Ronald Coase, Ronald Reagan, semantic web, shareholder value, Silicon Valley, The Chicago School, The Wealth of Nations by Adam Smith, trade route, transaction costs, trickle-down economics, web application, winner-take-all economy, zero-sum game
Moreover, as the Hargreaves review in the UK observed in May 2011, research has found “no evidence that changes since the launch of the original Napster ﬁle sharing in 1999/2000 have affected the quantity of new recorded music or artists coming to market.”34 In too many EFFECTIVE GLOBAL COPYRIGHT LAWS 261 cases, then, the problem arises from copyright owners’ lack of interest in fulﬁlling market demand. No law can cure this failure. But there is hope for the future. In a 2009 interview, MPAA Director of Special Projects Robert Bauer was quoted as saying, “Our job is to isolate the forms of piracy that compete with legitimate sales, treat those as a proxy for unmet consumer demand, and ﬁnd a way to meet that demand.”35 One way was recently suggested by Ben Karakunnel, director of business intelligence at Warner Brothers’ antipiracy unit, as reported in the paidContent. org website: In the international markets, illegal WB content in which pirates dub or subtitle themselves is increasingly popular. For one unspeciﬁed program Karakunnel used as an example, it wasn’t until the third day after its initial airdate that one such pirate-created translated version accounted for 23 percent of pirated ﬁles of that particular program.
Consent of the Networked: The Worldwide Struggle for Internet Freedom by Rebecca MacKinnon
A Declaration of the Independence of Cyberspace, Bay Area Rapid Transit, Berlin Wall, business intelligence, Cass Sunstein, Chelsea Manning, citizen journalism, cloud computing, cognitive dissonance, collective bargaining, conceptual framework, corporate social responsibility, Deng Xiaoping, digital Maoism, don't be evil, Filter Bubble, Firefox, future of journalism, illegal immigration, Jaron Lanier, Jeff Bezos, John Markoff, Julian Assange, Mark Zuckerberg, Mikhail Gorbachev, national security letter, online collectivism, Parag Khanna, pre–internet, race to the bottom, Richard Stallman, Ronald Reagan, sharing economy, Silicon Valley, Silicon Valley startup, Skype, Steve Crocker, Steven Levy, WikiLeaks
For more on DPI and its origins, see Hal Abelson, Ken Ledeen, and Chris Lewis, “Just Deliver the Packets,” in Essays on Deep Packet Inspection, Office of the Privacy Commissioner of Canada, 2009, http://dpi.priv.gc.ca/index.php/essays/just-deliver-the-packets; and Ralf Bendrath, “Global Technology Trends and National Regulation: Explaining Variation in the Governance of Deep Packet Inspection,” paper presented at the International Studies Annual Convention, New York City, February 2009, International Studies Association, http://userpage.fu-berlin.de/~bendrath/Paper_Ralf-Bendrath_DPI_v1-5.pdf. 59 the Egyptian government had purchased DPI technology from a company called Narus: Timothy Karr, “One US Corporation’s Role in Egypt’s Brutal Crackdown,” Huffington Post, January 28, 2011, www.huffingtonpost.com/timothy-karr/one-us-corporations-role-_b_815281.html (accessed June 27, 2011). 59 Narus signed a multimillion-dollar deal in 2005 with Giza Systems of Egypt: Trevor Lloyd-Jones, “Narus Signs Regional Licence with Giza Systems,” Business Intelligence Middle East, September 14, 2005, www.bi-me.com/main.php?id=2047&t=1 (accessed June 27, 2011). See also p. 10 of a Giza Systems newsletter, mentioning the licensing of Narus technology to Saudi Arabia and Libya: www.gizasystems.com/admin%5CNewsLetter%5CPDF%5C8.pdf. 59 Karr’s Free Press and the Paris-based Reporters Without Borders also lodged inquiries with Cisco Systems: Timothy Karr and Clothilde Le Coz, “Corporations and the Arab Net Crackdown,” March 25, 2011, www.fpif.org/articles/corporations_and_the_arab_net_crackdown (accessed June 27, 2011). 60 a business offer for an intrusion and surveillance software product called FinFisher: Eli Lake, “British Firm Offered Spy Software to Egypt,” Washington Times, April 25, 2011, www.washingtontimes.com/news/2011/apr/25/british-firm-offered-spy-software-to-egypt; Hussein uploaded the original document online: http://moftasa.posterous.com/finfisher-intrusion-software-spy-on-email-sof.
Forty Signs of Rain by Kim Stanley Robinson
bioinformatics, business intelligence, double helix, experimental subject, Intergovernmental Panel on Climate Change (IPCC), phenotype, prisoner's dilemma, Ronald Reagan, stem cell, the scientific method, zero-sum game
Bantam Books® is a registered trademark of Random House, Inc., and the colophon is a trademark of Random House, Inc. Visit our website at www.bantamdell.com Library of Congress Cataloging in Publication Data Robinson, Kim Stanley. Forty signs of rain / Kim Stanley Robinson. p. cm. eISBN 0-553-89817-5 1. Scientists—Fiction. 2. Legislators—Fiction. 3. Washington (D.C.)—Fiction. 4. Business intelligence—Fiction. I. Title. PS3568.O2893F67 2004 2003063683 v1.0 eBook Info Title: Forty Signs of Rain Creator: Kim Stanley Robinson Publisher: Random House Contributor: None Date: 2004-06-01 Format: OEB Identifier: 0-553-89817-5 Type: Subject: Description: Language: en Rights: Copyright © 2004 by Kim Stanley Robinson
Culture & Empire: Digital Revolution by Pieter Hintjens
4chan, airport security, anti-communist, anti-pattern, barriers to entry, Bill Duvall, bitcoin, blockchain, business climate, business intelligence, business process, Chelsea Manning, clean water, commoditize, congestion charging, Corn Laws, correlation does not imply causation, cryptocurrency, Debian, Edward Snowden, failed state, financial independence, Firefox, full text search, German hyperinflation, global village, GnuPG, Google Chrome, greed is good, Hernando de Soto, hiring and firing, informal economy, intangible asset, invisible hand, James Watt: steam engine, Jeff Rulifson, Julian Assange, Kickstarter, M-Pesa, mass immigration, mass incarceration, mega-rich, mutually assured destruction, Naomi Klein, national security letter, new economy, New Urbanism, Occupy movement, offshore financial centre, packet switching, patent troll, peak oil, pre–internet, private military company, race to the bottom, rent-seeking, reserve currency, RFC: Request For Comment, Richard Feynman, Richard Feynman, Richard Stallman, Satoshi Nakamoto, security theater, selection bias, Skype, slashdot, software patent, spectrum auction, Steve Crocker, Steve Jobs, Steven Pinker, Stuxnet, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, trade route, transaction costs, union organizing, wealth creators, web application, WikiLeaks, Y2K, zero day, Zipf's Law
According to Alex Constantine, author of "Mockingbird: The Subversion Of The Free Press By The CIA", in the 1950s, "some 3,000 salaried and contract CIA employees were eventually engaged in propaganda efforts." Officially, the program was ended in 1976 by incoming CIA Director George H. W. Bush. Inserting teams into existing media companies is one strategy. Another is to create your own business intelligence groups from the ground up. This is how large firms promote legislation, by funding "industry round tables" and "researchers" who push a pre-agreed message. The Spider has undoubtedly invested in many businesses, from armaments to drugs, and media. It's both profitable and convenient. Take as example The Economist, a respected and influential newspaper that was, ironically founded at the height of the patent debate in Britain as an anti-patent free-trade voice.
The Trade of Queens by Charles Stross
And then it was booby-trapped and staked out by the FTO. . "Stop right there!" Mike flipped the organizer open and turned to the address divider. "Paulet, Paulette, Powell-et? How do you spell it, it's a first name. . . ." He read for a long time, swearing occasionally at Miriam's spidery handwriting and her copious list of contacts—She's a journalist, it's what she does—until he hit paydirt a third of the way through: Milan, Paulette. Business intelligence division, the Weatherman. That was where Miriam had worked, last time he looked. "Bingo," Mike muttered. There was a cell number and a street address out in Somerville. He made a note of it; then, systematic to the end, he went back to the cassette tape. The next message was a call from Steve Schroeder—his voice familiar—asking Miriam to get in touch. It was followed by an odd double beep: some kind of tape position marker, probably.
The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies by Erik Brynjolfsson, Andrew McAfee
2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 3D printing, access to a mobile phone, additive manufacturing, Airbnb, Albert Einstein, Amazon Mechanical Turk, Amazon Web Services, American Society of Civil Engineers: Report Card, Any sufficiently advanced technology is indistinguishable from magic, autonomous vehicles, barriers to entry, basic income, Baxter: Rethink Robotics, British Empire, business intelligence, business process, call centre, Chuck Templeton: OpenTable, clean water, combinatorial explosion, computer age, computer vision, congestion charging, corporate governance, creative destruction, crowdsourcing, David Ricardo: comparative advantage, digital map, employer provided health coverage, en.wikipedia.org, Erik Brynjolfsson, factory automation, falling living standards, Filter Bubble, first square of the chessboard / second half of the chessboard, Frank Levy and Richard Murnane: The New Division of Labor, Freestyle chess, full employment, game design, global village, happiness index / gross national happiness, illegal immigration, immigration reform, income inequality, income per capita, indoor plumbing, industrial robot, informal economy, intangible asset, inventory management, James Watt: steam engine, Jeff Bezos, jimmy wales, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Joseph Schumpeter, Kevin Kelly, Khan Academy, knowledge worker, Kodak vs Instagram, law of one price, low skilled workers, Lyft, Mahatma Gandhi, manufacturing employment, Marc Andreessen, Mark Zuckerberg, Mars Rover, mass immigration, means of production, Narrative Science, Nate Silver, natural language processing, Network effects, new economy, New Urbanism, Nicholas Carr, Occupy movement, oil shale / tar sands, oil shock, pattern recognition, Paul Samuelson, payday loans, price stability, Productivity paradox, profit maximization, Ralph Nader, Ray Kurzweil, recommendation engine, Report Card for America’s Infrastructure, Robert Gordon, Rodney Brooks, Ronald Reagan, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Simon Kuznets, six sigma, Skype, software patent, sovereign wealth fund, speech recognition, statistical model, Steve Jobs, Steven Pinker, Stuxnet, supply-chain management, TaskRabbit, technological singularity, telepresence, The Bell Curve by Richard Herrnstein and Charles Murray, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, total factor productivity, transaction costs, Tyler Cowen: Great Stagnation, Vernor Vinge, Watson beat the top human players on Jeopardy!, winner-take-all economy, Y2K
“Computer and Dynamo: The Modern Productivity Paradox in a Not-Too-Distant Mirror,” Center for Economic Policy Research, no. 172, Stanford University, July 1989, http://www.dklevine.com/archive/refs4115.pdf. 10. For instance, Materials Resource Planning (MRP) systems, which begat Enterprise Resource Planning (ERP), and then Supply Chain Management (SCM), Customer Relationship Management (CRM), and, more recently, Business Intelligence (BI), Analytics and many other large-scale systems. 11. Todd Traub, “Wal-Mart Used Technology to Become Supply Chain Leader,” Arkansas Business, http://www.arkansasbusiness.com/article/85508/wal-mart-used-technology-to-become-supply-chain-leader (accessed July 20, 2013). 12. This is consistent with a similar analysis by Oliner and Sichel (2002), who wrote, “both the use of information technology and the efficiency gains associated with the production of information technology were central factors in [the productivity] resurgence.”
Python for Finance by Yuxing Yan
asset-backed security, business intelligence, capital asset pricing model, constrained optimization, correlation coefficient, distributed generation, diversified portfolio, implied volatility, market microstructure, P = NP, p-value, quantitative trading / quantitative ﬁnance, Sharpe ratio, time value of money, value at risk, volatility smile, zero-sum game
Rozario, Swati Kumari, Arwa Manasawala, Ruchita Bhansali, Apeksha Chitnis, and Pramila Balan as well as the external reviewers, Martin Olveyra, Mourad MOURAFIQ, and Loucas Parayiannis, for their valuable advice, suggestions, and criticism. Finally, and most importantly, I thank my wife, Xiaoning Jin, for her strong support, my daughter, Jing Yan, and son, James Yan, for their understanding and love they have showered on me over the years. About the Reviewers Jiri Pik is a finance and business intelligence consultant working with major investment banks, hedge funds, and other financial players. He has architected and delivered breakthrough trading, portfolio and risk management systems, and decision-support systems across industries. His consulting firm, WIXESYS, provides their clients with certified expertise, judgment, and execution at the speed of light. WIXESYS' power tools include revolutionary Excel and Outlook add-ons available at http://spearian.com.
1960s counterculture, Airbnb, business intelligence, Cass Sunstein, corporate governance, dematerialisation, experimental subject, Exxon Valdez, Frederick Winslow Taylor, Gini coefficient, income inequality, intangible asset, invisible hand, joint-stock company, lifelogging, market bubble, mental accounting, nudge unit, Philip Mirowski, profit maximization, randomized controlled trial, Richard Thaler, road to serfdom, Ronald Coase, Ronald Reagan, science of happiness, selective serotonin reuptake inhibitor (SSRI), sentiment analysis, sharing economy, Slavoj Žižek, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, Steve Jobs, The Chicago School, The Spirit Level, theory of mind, urban planning, Vilfredo Pareto
Over the next four years, it would open further offices in India, South Africa, Australia, Canada and Japan. Thanks to the security offered by its corporate behemoth of a patron, JWT became an international player, and its particular style of marketing expertise went global. The capacity of US businesses to export to global markets, which surged after World War Two, was greatly aided by the fact that such networks of business intelligence had already permeated much of the capitalist world. Knowledge of foreign consumers was already on hand. Following the acquisition of the GM contract, JWT set about accumulating consumer insight on an unprecedented scale. In less than eighteen months, over 44,000 interviews were done around the world, many in relation to cars, but also on topics such as food and toiletry consumption.2 This was the most ambitious project of mass psychological profiling ever attempted.
Rush Hour by Iain Gately
Albert Einstein, autonomous vehicles, Beeching cuts, blue-collar work, British Empire, business intelligence, business process, business process outsourcing, call centre, car-free, Cesare Marchetti: Marchetti’s constant, Clapham omnibus, cognitive dissonance, congestion charging, connected car, corporate raider, DARPA: Urban Challenge, Dean Kamen, decarbonisation, Deng Xiaoping, Detroit bankruptcy, don't be evil, Elon Musk, extreme commuting, Google bus, Henri Poincaré, Hyperloop, Jeff Bezos, low skilled workers, Marchetti’s constant, postnationalism / post nation state, Ralph Waldo Emerson, remote working, self-driving car, Silicon Valley, stakhanovite, Steve Jobs, telepresence, Tesla Model S, urban planning, éminence grise
Although the potential for lawsuits is dizzying, car-makers nonetheless bow to what they perceive to be their customers’ wants and compete to offer them better facilities for eating and drinking during their commutes. The automobile is now an official ‘Food and beverage venue’ in the opinion of Food and Drug Packaging, and home to a majority of the 4.4 billion ‘on-the-go’ ‘eating occasions’ that took place in the USA in 2008.*4 According to Datamonitor, the business intelligence provider, eating ‘is no longer considered a primary activity’ in the sense of being an end in itself, but rather is something we do while driving or working or watching television. Food on the go has become ‘ingrained in our lives’, and fast-food manufacturers have altered their products to suit the new Zeitgeist. ‘One-handed convenience’ is the contemporary mantra. A commuter should be able to grip a burger or taco in one hand without spilling bits of it into their lap.
Running Money by Andy Kessler
Andy Kessler, Apple II, bioinformatics, Bob Noyce, British Empire, business intelligence, buy low sell high, call centre, Corn Laws, Douglas Engelbart, family office, full employment, George Gilder, happiness index / gross national happiness, interest rate swap, invisible hand, James Hargreaves, James Watt: steam engine, joint-stock company, joint-stock limited liability company, knowledge worker, Leonard Kleinrock, Long Term Capital Management, mail merge, Marc Andreessen, margin call, market bubble, Maui Hawaii, Menlo Park, Metcalfe’s law, Network effects, packet switching, pattern recognition, pets.com, railway mania, risk tolerance, Robert Metcalfe, Sand Hill Road, Silicon Valley, South China Sea, spinning jenny, Steve Jobs, Steve Wozniak, Toyota Production System, zero-sum game
I made a note to myself to avoid NetZero. “I gotta go,” I told him. “I’ll bet Uncle Witty owned Quarter-deck, Level 3, 4-Systems, and the worst of them all, 3-Con.” “Take it easy,” I told Nick. “Some people never learn.” “Did you hear about Micro-tragedy? Could see that blowup from miles away?” It was Nick Moore again for my afternoon dose of cynicism. Microstrategy was one of those scary software blowups. They sold “business intelligence” software, one of those vague names only a McKinsey consultant could love, so companies could track business trends. The stock was hot, and growth was through the roof, until someone ﬁgured out that they were signing three-year contracts and reporting the revenues right away, a Bozo no-no of accounting. The stock imploded, the CEO not only ﬁred but forced to renege on some huge charitable donations he’d made with Microstrategy shares. 178 Running Money “What’s with these software companies?”
Bayesian statistics, business intelligence, business process, cellular automata, Celtic Tiger, cloud computing, collateralized debt obligation, conceptual framework, congestion charging, corporate governance, correlation does not imply causation, crowdsourcing, discrete time, George Gilder, Google Earth, Infrastructure as a Service, Internet Archive, Internet of things, invisible hand, knowledge economy, late capitalism, lifelogging, linked data, Masdar, means of production, Nate Silver, natural language processing, openstreetmap, pattern recognition, platform as a service, recommendation engine, RFID, semantic web, sentiment analysis, slashdot, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, smart grid, smart meter, software as a service, statistical model, supply-chain management, the scientific method, The Signal and the Noise by Nate Silver, transaction costs
In the case of lists, these are mostly rented and in many cases the renter does not receive the list, with a third-party service bureau preparing and sending mail on their behalf (CIPPIC 2006). Similarly, with data analytics, clients usually receive the analysis, but not the underlying data. Data consolidation and re-sale, and associated data analysis and value-added services, are a multibillion-dollar industry, with vast quantities of data and derived information being rented, bought and sold daily across a variety of markets – retail, financial, health, tourism, logistics, business intelligence, real estate, private security, political polling, and so on. These data concern all facets of everyday life and include public administration, communications, consumption of goods and media, travel, leisure, crime, social media interactions, and so on. Specialist data brokers have been around for a long time, collating data from media subscriptions (e.g., newspapers, magazines), mail-order retailers, polls, surveys, travel agencies, conferences, contests, product registration and warranties, payment processing companies, government records, and so on (CIPPIC 2006).
Albert Einstein, barriers to entry, Bayesian statistics, Berlin Wall, business intelligence, carbon-based life, Claude Shannon: information theory, complexity theory, David Heinemeier Hansson, declining real wages, deliberate practice, discrete time, double helix, Douglas Engelbart, Douglas Engelbart, Downton Abbey, Drosophila, Firefox, Frank Gehry, Google X / Alphabet X, informal economy, invention of the printing press, inventory management, John Markoff, Khan Academy, Kickstarter, low skilled workers, Lyft, Marc Andreessen, Mark Zuckerberg, meta analysis, meta-analysis, natural language processing, Network effects, open borders, pattern recognition, Peter Thiel, pez dispenser, ride hailing / ride sharing, Ronald Reagan, Ruby on Rails, Sand Hill Road, self-driving car, Silicon Valley, Silicon Valley startup, social web, South of Market, San Francisco, speech recognition, Steve Jobs, technoutopianism, transcontinental railway, Vannevar Bush
It turned out that women in different parts of the country have different preferences for bra colors and sizes. MicroStrategy analyzed the inventory data and told Victoria’s Secret it could make a lot more money by varying inventory accordingly. That kind of work was enough to make MicroStrategy a thriving technology company. Saylor played the venture capital game shrewdly, keeping the lion’s share of the company’s stock for himself. He also had grander visions than mere business intelligence. At MIT he had learned about the structure of scientific revolutions, how inventions like the printing press had profoundly changed not just the act of learning but whole systems of politics and philosophy. The world was on the verge of another such change, Saylor believed, and MicroStrategy software would be the key tool for making sense of the new information—not just for businesses but for every single person on earth.
SQL Hacks by Andrew Cumming, Gordon Russell
The wizard, shown in Figure 10-2, includes options to pull the source data from an ODBC source, and that includes pretty much every SQL product available. If you want to use this with MySQL, be sure to get the MySQL ODBC driver from http://dev.mysql.com/downloads/connector/odbc. Figure 10-2. PivotTable Wizard The wizard is intimidating at first, but with a little experimentation you can use it to produce useful business intelligence. The real value to this kind of report comes when the data is updated regularly and you can compare the most recent data with corresponding historical reports. This means that you have to produce the reports regularly; that is much easier if you automate the process. 10.1.1. Include Missing Values In Figure 10-1 you might notice that the location East is missing. By chance, no rows in the supplied data set involve a location of East, so Excel does not show the missing column.
Beautiful security by Andy Oram, John Viega
Albert Einstein, Amazon Web Services, business intelligence, business process, call centre, cloud computing, corporate governance, credit crunch, crowdsourcing, defense in depth, Donald Davies, en.wikipedia.org, fault tolerance, Firefox, loose coupling, Marc Andreessen, market design, Monroe Doctrine, new economy, Nicholas Carr, Nick Leeson, Norbert Wiener, optical character recognition, packet switching, peer-to-peer, performance metric, pirate software, Robert Bork, Search for Extraterrestrial Intelligence, security theater, SETI@home, Silicon Valley, Skype, software as a service, statistical model, Steven Levy, The Wisdom of Crowds, Upton Sinclair, web application, web of trust, x509 certificate, zero day, Zimmermann PGP
My money is on the mass collection, sharing, and analysis of information and benchmarking. Participants can run benchmarks of service quality in relation to software cost, corporate vulnerabilities, and threat profiles. The potential scope is enormous, as shown through the MeMe map in Figure 9-1, taken from a security industry source. Extensible (”hackable”) and tailorable Customer service; long tails and idea viruses Decentralized connectivity Trading platform Business intelligence Business peer groups Aligned to business goals Platform that connects people, process, and technology Information security as a business discipline Useful business-oriented services Focus on the whole space (information) and not just IT security Facilitate and allow modern business models (outsourcing) Business process management Genuinely useful Cost-effective, scalable, and fair Social networking where specialists openly add value Community drives content Workflow Community determines form via demand Open architecture for connectivity Fun and serious (“Wears it all well”) “Anti-widget” Design and style matter FIGURE 9-1.
AltaVista, barriers to entry, Black Swan, bounce rate, business intelligence, butterfly effect, call centre, Claude Shannon: information theory, complexity theory, correlation does not imply causation, en.wikipedia.org, first-price auction, information asymmetry, information retrieval, intangible asset, inventory management, life extension, linear programming, megacity, Nash equilibrium, Network effects, PageRank, place-making, price mechanism, psychological pricing, random walk, Schrödinger's Cat, sealed-bid auction, search engine result page, second-price auction, second-price sealed-bid, sentiment analysis, social web, software as a service, stochastic process, telemarketer, the market place, The Present Situation in Quantum Mechanics, the scientific method, The Wisdom of Crowds, Vickrey auction, Vilfredo Pareto, yield management
BTL can actually lead to a sale, and BTL promotions are highly measurable, giving marketers valuable insights into their return on investment (ROI) (Source: Wikipedia) (see Chapter BAM!). BtoB/B2B (Business-to-Business): businesses whose primary customers are other businesses (Source: IAB) (see Chapter 6 BAM!). BtoC/B2C (Business-to-Consumer): businesses whose primary customers are consumers (Source: IAB) (see Chapter 6 BAM!). Business intelligence: business practices that foster customer care, loyalty, and/or customer support. Also refers to a category of software and tools designed to gather, store, analyze, and deliver data in a user-friendly format to help organizations make more informed business decisions (Source: modified from WebTrends and IAB) (see Chapter 6 BAM!). Button: can have a couple of meanings depending on the context (Source: IAB) (see Chapter 2 model): • clickable graphic that contains certain functionality, such as taking one someplace or executing a program • buttons can also be ads.
Gnuplot in Action: Understanding Data With Graphs by Philipp Janert
bioinformatics, business intelligence, centre right, Debian, general-purpose programming language, iterative process, mandelbrot fractal, pattern recognition, random walk, Richard Stallman, six sigma, survivorship bias
About the author My education is in physics, and I’ve worked as technology consultant, software engineer, technical lead, and project manager, for small startups and in large corporate environments, both in the U.S. and overseas. I first started using gnuplot when I was a graduate student, and it has become an indispensable part of my toolbox: one of the handful of programs I can’t do without. Recently, I’ve also started to contribute a few features to the gnuplot development version. I provide consulting services specializing in corporate metrics, business intelligence, data analysis, and mathematical modeling through my company, Principal Value, LLC (www.principal-value.com). I also teach classes on software design and data analysis at the University of Washington. I hold a Ph.D. in theoretical physics from the University of Washington. Author online Purchase of Gnuplot in Action includes free access to a private web forum run by Manning Publications where you can make comments about the book, ask technical questions, and receive help from the author and from other users.
Airbnb, Amazon Web Services, barriers to entry, Black Swan, business intelligence, call centre, crowdsourcing, en.wikipedia.org, game design, Google Glasses, Google Hangouts, Google X / Alphabet X, iterative process, Jeff Bezos, Jony Ive, Kickstarter, knowledge worker, Lean Startup, loose coupling, Marc Andreessen, Mark Zuckerberg, minimum viable product, move fast and break things, move fast and break things, Network effects, Oculus Rift, Paul Graham, QR code, Ruby on Rails, self-driving car, Silicon Valley, Silicon Valley startup, Skype, Snapchat, social graph, software as a service, software is eating the world, Steve Jobs, Steven Levy, Y Combinator
As you drive towards becoming a robust, revenue-focused company, you need to ensure that you’re making more decisions based on data; you need to take advantage of all the information that you have at hand. While your off-the-shelf analytics solutions are pretty powerful, you will invariably hit a number of limitations. There’s a good chance these limitations are going to become very annoying and will start to affect business decisions. Business intelligence tools such as QlikView give you the ability to generate insight from all the data you have collected on the back-end and pull it quickly into dashboards and reports. Think of it as one level more powerful than your daily analytics tools, and a lot more user-friendly than manipulating raw data with SQL queries (SQL is a special programing language for managing and interrogating data that is stored in relational databases) – and yes, that’s meant to sound hard.
Connectography: Mapping the Future of Global Civilization by Parag Khanna
1919 Motor Transport Corps convoy, 2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 3D printing, 9 dash line, additive manufacturing, Admiral Zheng, affirmative action, agricultural Revolution, Airbnb, Albert Einstein, amateurs talk tactics, professionals talk logistics, Amazon Mechanical Turk, Asian financial crisis, asset allocation, autonomous vehicles, banking crisis, Basel III, Berlin Wall, bitcoin, Black Swan, blockchain, borderless world, Boycotts of Israel, Branko Milanovic, BRICs, British Empire, business intelligence, call centre, capital controls, charter city, clean water, cloud computing, collateralized debt obligation, commoditize, complexity theory, continuation of politics by other means, corporate governance, corporate social responsibility, credit crunch, crony capitalism, crowdsourcing, cryptocurrency, cuban missile crisis, data is the new oil, David Ricardo: comparative advantage, deglobalization, deindustrialization, dematerialisation, Deng Xiaoping, Detroit bankruptcy, digital map, diversification, Doha Development Round, edge city, Edward Snowden, Elon Musk, energy security, ethereum blockchain, European colonialism, eurozone crisis, failed state, Fall of the Berlin Wall, family office, Ferguson, Missouri, financial innovation, financial repression, fixed income, forward guidance, global supply chain, global value chain, global village, Google Earth, Hernando de Soto, high net worth, Hyperloop, ice-free Arctic, if you build it, they will come, illegal immigration, income inequality, income per capita, industrial cluster, industrial robot, informal economy, Infrastructure as a Service, interest rate swap, Intergovernmental Panel on Climate Change (IPCC), Internet of things, Isaac Newton, Jane Jacobs, Jaron Lanier, John von Neumann, Julian Assange, Just-in-time delivery, Kevin Kelly, Khyber Pass, Kibera, Kickstarter, labour market flexibility, labour mobility, LNG terminal, low cost carrier, manufacturing employment, mass affluent, mass immigration, megacity, Mercator projection, Metcalfe’s law, microcredit, mittelstand, Monroe Doctrine, mutually assured destruction, New Economic Geography, new economy, New Urbanism, off grid, offshore financial centre, oil rush, oil shale / tar sands, oil shock, openstreetmap, out of africa, Panamax, Parag Khanna, Peace of Westphalia, peak oil, Pearl River Delta, Peter Thiel, Philip Mirowski, Plutocrats, plutocrats, post-oil, post-Panamax, private military company, purchasing power parity, QWERTY keyboard, race to the bottom, Rana Plaza, rent-seeking, reserve currency, Robert Gordon, Robert Shiller, Robert Shiller, Ronald Coase, Scramble for Africa, Second Machine Age, sharing economy, Shenzhen was a fishing village, Silicon Valley, Silicon Valley startup, six sigma, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, South China Sea, South Sea Bubble, sovereign wealth fund, special economic zone, spice trade, Stuxnet, supply-chain management, sustainable-tourism, TaskRabbit, telepresence, the built environment, The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, Tim Cook: Apple, trade route, transaction costs, UNCLOS, uranium enrichment, urban planning, urban sprawl, WikiLeaks, young professional, zero day
ESRI http://storymaps.arcgis.com/en/ Esri’s Story Map apps can be customized to produce thematic visual stories such as how rapid urban migration has given rise to a world of megacities. ESRI MAPPING CENTER http://mappingcenter.esri.com/index.cfm?fa=resources.cartoFavorites Esri’s Mapping Center provides access to various resources that are used regularly by professional mapmakers and cartographers, enabling its users to create maps using ArcGIS. FIRST MILE GEO https://www.firstmilegeo.com/ First Mile Geo is a business intelligence software that enables users to collect, visualize, and monitor data collected online or off-line through mobile, SMS, surveys, or manual sources. Maps, dashboards, indices, and alerts can be generated in multiple languages. FLEETMON http://www.fleetmon.com/live_tracking/fleetmon_explorer FleetMon is an open database of ships and ports worldwide that uses real-time AIS positioning data to visualize the location and movement of nearly 500,000 vessels, allowing for the analysis of shipping and trade patterns.
Rise of the Machines: A Cybernetic History by Thomas Rid
1960s counterculture, A Declaration of the Independence of Cyberspace, agricultural Revolution, Albert Einstein, Alistair Cooke, Apple II, Apple's 1984 Super Bowl advert, back-to-the-land, Berlin Wall, British Empire, Brownian motion, Buckminster Fuller, business intelligence, Claude Shannon: information theory, conceptual framework, connected car, domain-specific language, Douglas Engelbart, Douglas Engelbart, dumpster diving, Extropian, full employment, game design, global village, Haight Ashbury, Howard Rheingold, Jaron Lanier, job automation, John Markoff, John von Neumann, Kevin Kelly, Marshall McLuhan, Menlo Park, Mother of all demos, new economy, New Journalism, Norbert Wiener, offshore financial centre, oil shale / tar sands, pattern recognition, RAND corporation, Silicon Valley, Simon Singh, speech recognition, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, technoutopianism, Telecommunications Act of 1996, telepresence, V2 rocket, Vernor Vinge, Whole Earth Catalog, Whole Earth Review, Y2K, Yom Kippur War, Zimmermann PGP
“Any other juicy stuff is always welcome,” the anonymous voice added. BlackNet was specifically interested in buying trade secrets (“semiconductors”); production methods in nanotechnology (“the Merkle sleeve bearing”); chemical manufacturing (“fullerines and protein folding”); and design plans for things ranging from children’s toys to cruise missiles (“3DO”). Oh, and BlackNet was also interested in general business intelligence—for instance, on mergers and buyouts. “Join us in this revolutionary—and profitable—venture,” the message concluded.62 May soon claimed credit for the letter. He had come up with BlackNet in the summer of 1993, as an example of what could be done, “an exercise in guerrilla ontology,” as he called it.63 The goal was to find a way to ensure fully anonymous, untraceable, two-way exchanges of information.
Take the money and run: sovereign wealth funds and the demise of American prosperity by Eric Curt Anderson
asset allocation, banking crisis, Bretton Woods, business continuity plan, business intelligence, business process, collective bargaining, corporate governance, credit crunch, currency manipulation / currency intervention, currency peg, diversified portfolio, fixed income, floating exchange rates, housing crisis, index fund, Kenneth Rogoff, open economy, passive investing, profit maximization, profit motive, random walk, reserve currency, risk tolerance, risk-adjusted returns, risk/return, Ronald Reagan, sovereign wealth fund, the market place, The Wealth of Nations by Adam Smith, too big to fail, Vanguard fund
Neil Irwin and Amit Paley, “Greenspan Says He Was Wrong on Regulation,” Washington Post, 24 October 2008. 5. Warren Buffett, comments made while being interviewed on CNBC, 26 December 2007. 6. Heather Timmons and Keith Bradsher, “To Avoid Risk and Diversify, Sovereign Funds Move on from Banks,” New York Times, 19 September 2008. 7. “Kuwait Wealth Fund is not ‘Responsible’ for Saving Banks,” BusinessIntelligence Middle East, bi-me.com, 23 September 2008. 8. Jason Dean, Yuka Hayashi, Alison Tudor, and Rick Carew, “Caution, Inexperience Limit Extent of Asia’s Newfound Clout in Crisis,” The Wall Street Journal, 6 October 2008. 9. Ibid. 10. Ellen Knickmeyer and Faiza Saleh Ambah, “Gulf States Lose Their Swagger Amid Regionwide Sell-Off,” Washington Post, 9 October 2008. 11. Tony Jackson, “Sovereign Wealth Funds Appear to have Lost Their Way,” Financial Times, London, 7 September 2008. 12.
Radical Technologies: The Design of Everyday Life by Adam Greenfield
3D printing, Airbnb, augmented reality, autonomous vehicles, bank run, barriers to entry, basic income, bitcoin, blockchain, business intelligence, business process, call centre, cellular automata, centralized clearinghouse, centre right, Chuck Templeton: OpenTable, cloud computing, collective bargaining, combinatorial explosion, Computer Numeric Control, computer vision, Conway's Game of Life, cryptocurrency, David Graeber, dematerialisation, digital map, distributed ledger, drone strike, Elon Musk, ethereum blockchain, facts on the ground, fiat currency, global supply chain, global village, Google Glasses, IBM and the Holocaust, industrial robot, informal economy, information retrieval, Internet of things, James Watt: steam engine, Jane Jacobs, Jeff Bezos, job automation, John Conway, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, joint-stock company, Kevin Kelly, Kickstarter, late capitalism, license plate recognition, lifelogging, M-Pesa, Mark Zuckerberg, means of production, megacity, megastructure, minimum viable product, money: store of value / unit of account / medium of exchange, natural language processing, Network effects, New Urbanism, Occupy movement, Oculus Rift, Pareto efficiency, pattern recognition, Pearl River Delta, performance metric, Peter Eisenman, Peter Thiel, planetary scale, Ponzi scheme, post scarcity, RAND corporation, recommendation engine, RFID, rolodex, Satoshi Nakamoto, self-driving car, sentiment analysis, shareholder value, sharing economy, Silicon Valley, smart cities, smart contracts, sorting algorithm, special economic zone, speech recognition, stakhanovite, statistical model, stem cell, technoutopianism, Tesla Model S, the built environment, The Death and Life of Great American Cities, The Future of Employment, transaction costs, Uber for X, universal basic income, urban planning, urban sprawl, Whole Earth Review, WikiLeaks, women in the workforce
The fashion executive had her assistant book an Uber for her; while there’s certainly something to be inferred from the fact that she splurged on the Mercedes as usual instead of economizing with a cheaper booking, there’s still some question as to whether this signifies her own impression of her status, or the assistant’s. Even if the car hadn’t been booked on the corporate account, it is also equipped with GPS, and that unit’s accuracy buffer has been set such that it correctly identifies the location at the moment it pulls up to the curb, and tags the booking with the name of the house the executive works for. Here can be gleaned solid, actionable business intelligence: both the cycling of particular enterprises and sectors of the economy, and by extension possibly even some insight into the rhythms of something as inchoate and hard to grasp as taste. What might we learn from the American backpacker? The pedometer app on his phone is sophisticated enough to understand his pause of eleven minutes in a location in the Rue de Rivoli as a visit to the WHSmith bookstore, but other facets of his activity through the day slip through holes in the mesh.
The Singularity Is Near: When Humans Transcend Biology by Ray Kurzweil
additive manufacturing, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anthropic principle, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, Benoit Mandelbrot, Bill Joy: nanobots, bioinformatics, brain emulation, Brewster Kahle, Brownian motion, business intelligence, c2.com, call centre, carbon-based life, cellular automata, Claude Shannon: information theory, complexity theory, conceptual framework, Conway's Game of Life, cosmological constant, cosmological principle, cuban missile crisis, data acquisition, Dava Sobel, David Brooks, Dean Kamen, disintermediation, double helix, Douglas Hofstadter, en.wikipedia.org, epigenetics, factory automation, friendly AI, George Gilder, Gödel, Escher, Bach, informal economy, information retrieval, invention of the telephone, invention of the telescope, invention of writing, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, job automation, job satisfaction, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, lifelogging, linked data, Loebner Prize, Louis Pasteur, mandelbrot fractal, Mikhail Gorbachev, mouse model, Murray Gell-Mann, mutually assured destruction, natural language processing, Network effects, new economy, Norbert Wiener, oil shale / tar sands, optical character recognition, pattern recognition, phenotype, premature optimization, randomized controlled trial, Ray Kurzweil, remote working, reversible computing, Richard Feynman, Richard Feynman, Robert Metcalfe, Rodney Brooks, Search for Extraterrestrial Intelligence, selection bias, semantic web, Silicon Valley, Singularitarianism, speech recognition, statistical model, stem cell, Stephen Hawking, Stewart Brand, strong AI, superintelligent machines, technological singularity, Ted Kaczynski, telepresence, The Coming Technological Singularity, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Y2K, Yogi Berra
I have been inundated with thousands of compelling examples. In our reporting on the KurzweilAI.net Web site, we feature one or more dramatic systems almost every day.180 A 2003 study by Business Communications Company projected a $21 billion market by 2007 for AI applications, with average annual growth of 12.2 percent from 2002 to 2007.181 Leading industries for AI applications include business intelligence, customer relations, finance, defense and domestic security, and education. Here is a small sample of narrow AI in action. Military and Intelligence. The U.S. military has been an avid user of AI systems. Pattern-recognition software systems guide autonomous weapons such as cruise missiles, which can fly thousands of miles to find a specific building or even a specific window.182 Although the relevant details of the terrain that the missile flies over are programmed ahead of time, variations in weather, ground cover, and other factors require a flexible level of real-time image recognition.
Masterminds of Programming: Conversations With the Creators of Major Programming Languages by Federico Biancuzzi, Shane Warden
Benevolent Dictator For Life (BDFL), business intelligence, business process, cellular automata, cloud computing, commoditize, complexity theory, conceptual framework, continuous integration, data acquisition, domain-specific language, Douglas Hofstadter, Fellow of the Royal Society, finite state, Firefox, follow your passion, Frank Gehry, general-purpose programming language, Guido van Rossum, HyperCard, information retrieval, iterative process, John von Neumann, Larry Wall, linear programming, loose coupling, Mars Rover, millennium bug, NP-complete, Paul Graham, performance metric, Perl 6, QWERTY keyboard, RAND corporation, randomized controlled trial, Renaissance Technologies, Ruby on Rails, Sapir-Whorf hypothesis, Silicon Valley, slashdot, software as a service, software patent, sorting algorithm, Steve Jobs, traveling salesman, Turing complete, type inference, Valgrind, Von Neumann architecture, web application
The SQL standard has also served to focus the industry’s attention and resources, providing a common framework in which individuals and companies could develop tools, write books, teach courses, and provide consulting services. SQL manages persistent data, which has a long lifetime. Enterprises that have an investment in SQL databases are inclined to build on that investment rather than starting over with a different approach. SQL is robust enough to solve real problems. It spans a broad spectrum of usage from business intelligence to transaction processing. It is supported on many platforms and in many processing environments. Despite some criticism about its lack of elegance, SQL has been used successfully by many organizations to develop critical, real-world applications. I believe that this success reflects the origin of the language in an experimental prototype that was responsive to the needs of real users from the earliest days.
Natural language processing with Python by Steven Bird, Ewan Klein, Edward Loper
bioinformatics, business intelligence, conceptual framework, Donald Knuth, elephant in my pajamas, en.wikipedia.org, finite state, Firefox, Guido van Rossum, information retrieval, Menlo Park, natural language processing, P = NP, search inside the book, speech recognition, statistical model, text mining, Turing test
In this chapter we take a different approach, deciding in advance that we will only look for very specific kinds of information in text, such as the relation between organizations and locations. Rather than trying to use text like (1) to answer the question directly, we first convert the unstructured data of natural language sentences into the structured data of Table 7-1. Then we reap the benefits of powerful query tools such as SQL. This method of getting meaning from text is called Information Extraction. Information Extraction has many applications, including business intelligence, resume harvesting, media analysis, sentiment detection, patent search, and email scanning. A particularly important area of current research involves the attempt to extract 262 | Chapter 7: Extracting Information from Text structured data out of electronically available scientific literature, especially in the domain of biology and medicine. Information Extraction Architecture Figure 7-1 shows the architecture for a simple information extraction system.
Hadoop: The Definitive Guide by Tom White
Amazon Web Services, bioinformatics, business intelligence, combinatorial explosion, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, full text search, Grace Hopper, information retrieval, Internet Archive, linked data, loose coupling, openstreetmap, recommendation engine, RFID, SETI@home, social graph, web application
Today, Hive is a successful Apache project used by many organizations as a general-purpose, scalable data processing platform. Of course, SQL isn’t ideal for every big data problem—it’s not a good fit for building complex machine learning algorithms, for example—but it’s great for many analyses, and it has the huge advantage of being very well known in the industry. What’s more, SQL is the lingua franca in business intelligence tools (ODBC is a common bridge, for example), so Hive is well placed to integrate with these products. This chapter is an introduction to using Hive. It assumes that you have working knowledge of SQL and general database architecture; as we go through Hive’s features, we’ll often compare them to the equivalent in a traditional RDBMS. Installing Hive In normal use, Hive runs on your workstation and converts your SQL query into a series of MapReduce jobs for execution on a Hadoop cluster.
Albert Einstein, British Empire, business intelligence, centralized clearinghouse, City Beautiful movement, estate planning, glass ceiling, In Cold Blood by Truman Capote, indoor plumbing, Livingstone, I presume, new economy, Plutocrats, plutocrats, refrigerator car, transcontinental railway, traveling salesman, women in the workforce, Works Progress Administration, young professional
A former president of the American Bankers Association and adviser to Presidents Taft and Hoover, Swinney had put together the deal for Kansas City Union Station, and he still sat on the boards of several major companies. The aging banker had been a friend and close colleague of Ford Harvey—who sat on his board and worked with him on the rebuilding of Kansas City’s streetcar system. He had also known Byron for years. So, when E. F. Swinney said that he held Kitty Harvey’s business intelligence in high esteem—and that, in fact, he thought she had a better business brain than her uncle—people listened. Of course, Swinney had his own civic reasons for keeping Fred Harvey in Kansas City. But he also had spent a lot of time with Kitty as she and Freddie worked at investing the money from their father’s estate. He was impressed with her. She was an agile, creative investor. Besides stocks and bonds, she would invest in the work of inventors and sell at just the right time.
Piracy : The Intellectual Property Wars from Gutenberg to Gates by Adrian Johns
active measures, banking crisis, Berlin Wall, British Empire, Buckminster Fuller, business intelligence, commoditize, Corn Laws, demand response, distributed generation, Douglas Engelbart, Douglas Engelbart, Edmond Halley, Ernest Rutherford, Fellow of the Royal Society, full employment, Hacker Ethic, Howard Rheingold, informal economy, invention of the printing press, Isaac Newton, James Watt: steam engine, John Harrison: Longitude, Marshall McLuhan, Mont Pelerin Society, new economy, New Journalism, Norbert Wiener, pirate software, Republic of Letters, Richard Stallman, road to serfdom, Ronald Coase, software patent, South Sea Bubble, Steven Levy, Stewart Brand, Ted Nelson, the scientific method, traveling salesman, Whole Earth Catalog
But nothing resembling their drilled “commandoes” had ever been put into the field before. They took this initiative at the very moment when private detective agencies such as Pinkerton’s were coming into their own in America and Britain as entrepreneurial counterparts (and sometimes more) to the professionalizing police forces. Like them, Preston and Abbott had hit upon an opportunity that was not to go away. Their initiative marked the beginning of an alliance between business, intelligence, policing, and intellectual property that would endure long after their victory. Today, private antipiracy policing is a growth industry. It recruits expolicemen, as Preston and Abbott did, and it too has been known to pursue its quarry not just as pirates but as criminal conspirators. The modern pirate hunters propel policy and legislation too. In the early years of the twentieth century, the private antipiracy police raised serious questions about everyday rights and freedoms – questions that many at the time, including prominent legal officers, viewed as seriously as any concerning piracy itself.
Solr in Action by Trey Grainger, Timothy Potter
business intelligence, cloud computing, commoditize, conceptual framework, crowdsourcing, data acquisition, en.wikipedia.org, failed state, fault tolerance, finite state, full text search, glass ceiling, information retrieval, natural language processing, performance metric, premature optimization, recommendation engine, web application
The NoSQL feature set was also expanded to include transaction logs, update durability, optimistic concurrency, and atomic updates. Solr in Action, written by longtime Solr power users and community members, Trey and Timothy, covers these important recent Solr features and provides an excellent starting point for those new to Solr. Solr is now used in more places than I could ever have imagined—from integrated library systems to e-commerce platforms, analytics and business intelligence products, content-management systems, internet searches, and more. It’s been rewarding to see Solr grow from a few early adopters to a huge global community of helpful users and active volunteers cooperatively pushing development forward. Solr in Action gives you the knowledge and techniques you need to use Solr’s features that have been under development since 2004. With Solr in Action in hand, you too are now well equipped to join the global community and help take Solr to new heights!
Site Reliability Engineering by Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy
Air France Flight 447, anti-pattern, barriers to entry, business intelligence, business process, Checklist Manifesto, cloud computing, combinatorial explosion, continuous integration, correlation does not imply causation, crowdsourcing, database schema, defense in depth, DevOps, en.wikipedia.org, fault tolerance, Flash crash, George Santayana, Google Chrome, Google Earth, job automation, job satisfaction, linear programming, load shedding, loose coupling, meta analysis, meta-analysis, minimum viable product, MVC pattern, performance metric, platform as a service, revision control, risk tolerance, side project, six sigma, the scientific method, Toyota Production System, trickle-down economics, web application, zero day
Glass, Facts and Fallacies of Software Engineering, Addison-Wesley Professional, 2002. [Gol14] W. Golab et al., “Eventually Consistent: Not What You Were Expecting?”, in ACM Queue, vol. 12, no. 1, 2014. [Gra09] P. Graham, “Maker’s Schedule, Manager’s Schedule”, blog post, July 2009. [Gup15] A. Gupta and J. Shute, “High-Availability at Massive Scale: Building Google’s Data Infrastructure for Ads”, in Workshop on Business Intelligence for the Real Time Enterprise, 2015. [Ham07] J. Hamilton, “On Designing and Deploying Internet-Scale Services”, in Proceedings of the 21st Large Installation System Administration Conference, November 2007. [Han94] S. Hanks, T. Li, D. Farinacci, and P. Traina, “Generic Routing Encapsulation over IPv4 networks”, IETF Informational RFC, 1994. [Hic11] M. Hickins, “Tape Rescues Google in Lost Email Scare”, in Digits, Wall Street Journal, 1 March 2011.
The Art of Scalability: Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise by Martin L. Abbott, Michael T. Fisher
always be closing, anti-pattern, barriers to entry, Bernie Madoff, business climate, business continuity plan, business intelligence, business process, call centre, cloud computing, combinatorial explosion, commoditize, Computer Numeric Control, conceptual framework, database schema, discounted cash flows, en.wikipedia.org, fault tolerance, finite state, friendly fire, hiring and firing, Infrastructure as a Service, inventory management, new economy, packet switching, performance metric, platform as a service, Ponzi scheme, RFC: Request For Comment, risk tolerance, Rubik’s Cube, Search for Extraterrestrial Intelligence, SETI@home, shareholder value, Silicon Valley, six sigma, software as a service, the scientific method, transaction costs, Vilfredo Pareto, web application, Y2K
We guarantee that scalability was on the mind of every bank during the consolidation that occurred after the collapse of the banking industry. The models and approaches that we present in our book are industry agnostic. They have been developed, tested, and proven successful in some of the fastest growing companies of our time and they work not only in front-end customer facing transaction processing systems but back-end business intelligence, enterprise resource planning, and customer relationship management systems. They don’t discriminate by activity, but rather help to guide the thought process on how to separate systems, organizations, and processes to meet the objective of becoming highly scalable and reaching a level of scale that allows your business to operate without concerns regarding customer or end-user demand. Book Organization and Structure We’ve divided the book into four parts.
Backup & Recovery by W. Curtis Preston
Berlin Wall, business intelligence, business process, database schema, Debian, dumpster diving, failed state, fault tolerance, full text search, job automation, side project, Silicon Valley, web application
SQL Server 2005, the currently shipping version at this writing, has grown beyond the standard basic database package that includes only the database engine. SQL Server 2005 also offers a wealth of additional features such as services for analysis, data integration, notification, and reporting as well as the service broker. Combined, these elements make up a complete relational database system that can be used for simple tasks such as a database-driven web application or for more advanced needs such as data mining, complex business intelligence gathering, specialized reporting and notification, and a host of additional needs. Tip When referring to specific versions, this chapter often uses just the version numbers. For example, 2000 refers to SQL Server 2000, and 2005 refers to SQL Server 2005. When referring to both SQL Server 2000 and 2005, I use the term SQL Server. Overview of SQL Server As with any piece of complex software, SQL Server comes in multiple configurations.