THE FINANCIAL CRASH OF 2007 – 2008 – ITS GLOBAL SOCIO ECONOMIC IMPACT

12 22 2011

Barbara & John Ehrenreich - The Making of the American 99% and the Collapse of The Middle Class
By Barbara Ehrenreich and John Ehrenreich


“Class happens when some men, as a result of common experiences (inherited or shared), feel and articulate the identity of their interests as between themselves, and as against other men whose interests are different from (and usually opposed to) theirs.”


-- E.P. Thompson, The Making of the English Working Class


The “other men” (and of course women) in the current American class alignment are those in the top 1% of the wealth distribution -- the bankers, hedge-fund managers, and CEOs targeted by the Occupy Wall Street movement. They have been around for a long time in one form or another, but they only began to emerge as a distinct and visible group, informally called the “super-rich,” in recent years.


Extravagant levels of consumption helped draw attention to them: private jets, multiple 50,000 square-foot mansions, $25,000 chocolate desserts embellished with gold dust. But as long as the middle class could still muster the credit for college tuition and occasional home improvements, it seemed churlish to complain. Then came the financial crash of 2007-2008, followed by the Great Recession, and the 1% to whom we had entrusted our pensions, our economy, and our political system stood revealed as a band of feckless, greedy narcissists, and possibly sociopaths.


Still, until a few months ago, the 99% was hardly a group capable of (as Thompson says) articulating “the identity of their interests.” It contained, and still contains, most “ordinary” rich people, along with middle-class professionals, factory workers, truck drivers, and miners, as well as the much poorer people who clean the houses, manicure the fingernails, and maintain the lawns of the affluent.


It was divided not only by these class differences, but most visibly by race and ethnicity -- a division that has actually deepened since 2008. African-Americans and Latinos of all income levels disproportionately lost their homes to foreclosure in 2007 and 2008, and then disproportionately lost their jobs in the wave of layoffs that followed. On the eve of the Occupy movement, the black middle class had been devastated. In fact, the only political movements to have come out of the 99% before Occupy emerged were the Tea Party movement and, on the other side of the political spectrum, the resistance to restrictions on collective bargaining in Wisconsin.


But Occupy could not have happened if large swaths of the 99% had not begun to discover some common interests, or at least to put aside some of the divisions among themselves. For decades, the most stridently promoted division within the 99% was the one between what the right calls the “liberal elite” -- composed of academics, journalists, media figures, etc. -- and pretty much everyone else.


As Harper’s Magazine columnist Tom Frank has brilliantly explained, the right earned its spurious claim to populism by targeting that “liberal elite,” which supposedly favors reckless government spending that requires oppressive levels of taxes, supports “redistributive” social policies and programs that reduce opportunity for the white middle class, creates ever more regulations (to, for instance, protect the environment) that reduce jobs for the working class, and promotes kinky countercultural innovations like gay marriage. The liberal elite, insisted conservative intellectuals, looked down on “ordinary” middle- and working-class Americans, finding them tasteless and politically incorrect. The “elite” was the enemy, while the super-rich were just like everyone else, only more “focused” and perhaps a bit better connected.


Of course, the “liberal elite” never made any sociological sense. Not all academics or media figures are liberal (Newt Gingrich, George Will, Rupert Murdoch). Many well-educated middle managers and highly trained engineers may favor latte over Red Bull, but they were never targets of the right. And how could trial lawyers be members of the nefarious elite, while their spouses in corporate law firms were not?


A Greased Chute, Not a Safety Net


“Liberal elite” was always a political category masquerading as a sociological one. What gave the idea of a liberal elite some traction, though, at least for a while, was that the great majority of us have never knowingly encountered a member of the actual elite, the 1% who are, for the most part, sealed off in their own bubble of private planes, gated communities, and walled estates.


The authority figures most people are likely to encounter in their daily lives are teachers, doctors, social workers, and professors. These groups (along with middle managers and other white-collar corporate employees) occupy a much lower position in the class hierarchy. They made up what we described in a 1976 essay as the “professional managerial class.” As we wrote at the time, on the basis of our experience of the radical movements of the 1960s and 1970s, there have been real, longstanding resentments between the working-class and middle-class professionals. These resentments, which the populist right cleverly deflected toward “liberals,” contributed significantly to that previous era of rebellion’s failure to build a lasting progressive movement.


As it happened, the idea of the “liberal elite” could not survive the depredations of the 1% in the late 2000s. For one thing, it was summarily eclipsed by the discovery of the actual Wall Street-based elite and their crimes. Compared to them, professionals and managers, no matter how annoying, were pikers. The doctor or school principal might be overbearing, the professor and the social worker might be condescending, but only the 1% took your house away.


There was, as well, another inescapable problem embedded in the right-wing populist strategy: even by 2000, and certainly by 2010, the class of people who might qualify as part of the “liberal elite” was in increasingly bad repair. Public-sector budget cuts and corporate-inspired reorganizations were decimating the ranks of decently paid academics, who were being replaced by adjunct professors working on bare subsistence incomes. Media firms were shrinking their newsrooms and editorial budgets. Law firms had started outsourcing their more routine tasks to India. Hospitals beamed X-rays to cheap foreign radiologists. Funding had dried up for nonprofit ventures in the arts and public service. Hence the iconic figure of the Occupy movement: the college graduate with tens of thousands of dollars in student loan debts and a job paying about $10 a hour, or no job at all.


These trends were in place even before the financial crash hit, but it took the crash and its grim economic aftermath to awaken the 99% to a widespread awareness of shared danger. In 2008, “Joe the Plumber’s” intention to earn a quarter-million dollars a year still had some faint sense of plausibility. A couple of years into the recession, however, sudden downward mobility had become the mainstream American experience, and even some of the most reliably neoliberal media pundits were beginning to announce that something had gone awry with the American dream.


Once-affluent people lost their nest eggs as housing prices dropped off cliffs. Laid-off middle-aged managers and professionals were staggered to find that their age made them repulsive to potential employers. Medical debts plunged middle-class households into bankruptcy. The old conservative dictum -- that it was unwise to criticize (or tax) the rich because you might yourself be one of them someday -- gave way to a new realization that the class you were most likely to migrate into wasn’t the rich, but the poor.


And here was another thing many in the middle class were discovering: the downward plunge into poverty could occur with dizzying speed. One reason the concept of an economic 99% first took root in America rather than, say, Ireland or Spain is that Americans are particularly vulnerable to economic dislocation. We have little in the way of a welfare state to stop a family or an individual in free-fall. Unemployment benefits do not last more than six months or a year, though in a recession they are sometimes extended by Congress. At present, even with such an extension, they reach only about half the jobless. Welfare was all but abolished 15 years ago, and health insurance has traditionally been linked to employment.


In fact, once an American starts to slip downward, a variety of forces kick in to help accelerate the slide. An estimated 60% of American firms now check applicants' credit ratings, and discrimination against the unemployed is widespread enough to have begun to warrant Congressional concern. Even bankruptcy is a prohibitively expensive, often crushingly difficult status to achieve. Failure to pay government-imposed fines or fees can even lead, through a concatenation of unlucky breaks, to an arrest warrant or a criminal record. Where other once-wealthy nations have a safety net, America offers a greased chute, leading down to destitution with alarming speed.


Making Sense of the 99%


The Occupation encampments that enlivened approximately 1,400 cities this fall provided a vivid template for the 99%’s growing sense of unity. Here were thousands of people -- we may never know the exact numbers -- from all walks of life, living outdoors in the streets and parks, very much as the poorest of the poor have always lived: without electricity, heat, water, or toilets. In the process, they managed to create self-governing communities.


General assembly meetings brought together an unprecedented mix of recent college graduates, young professionals, elderly people, laid-off blue-collar workers, and plenty of the chronically homeless for what were, for the most part, constructive and civil exchanges. What started as a diffuse protest against economic injustice became a vast experiment in class building. The 99%, which might have seemed to be a purely aspirational category just a few months ago, began to will itself into existence.


Can the unity cultivated in the encampments survive as the Occupy movement evolves into a more decentralized phase? All sorts of class, racial, and cultural divisions persist within that 99%, including distrust between members of the former “liberal elite” and those less privileged. It would be surprising if they didn’t. The life experience of a young lawyer or a social worker is very different from that of a blue-collar worker whose work may rarely allow for biological necessities like meal or bathroom breaks. Drum circles, consensus decision-making, and masks remain exotic to at least the 90%. “Middle class” prejudice against the homeless, fanned by decades of right-wing demonization of the poor, retains much of its grip.


Sometimes these differences led to conflict in Occupy encampments -- for example, over the role of the chronically homeless in Portland or the use of marijuana in Los Angeles -- but amazingly, despite all the official warnings about health and safety threats, there was no “Altamont moment”: no major fires and hardly any violence. In fact, the encampments engendered almost unthinkable convergences: people from comfortable backgrounds learning about street survival from the homeless, a distinguished professor of political science discussing horizontal versus vertical decision-making with a postal worker, military men in dress uniforms showing up to defend the occupiers from the police.


Class happens, as Thompson said, but it happens most decisively when people are prepared to nourish and build it. If the “99%” is to become more than a stylish meme, if it’s to become a force to change the world, eventually we will undoubtedly have to confront some of the class and racial divisions that lie within it. But we need to do so patiently, respectfully, and always with an eye to the next big action -- the next march, or building occupation, or foreclosure fight, as the situation demands.


By Barbara Ehrenreich and John Ehrenreich
Posted on December 15, 2011
http://www.tomdispatch.com/blog/175480/

WATCHDOG WARNS CANADA GOVERNMENTS OF FISCAL CRUNCH

OTTAWA (Reuters) - The finances of Canada's federal government and its 10 provinces are unsustainable over the long term and they will need to either raise taxes or cut spending, in part because the population is aging, the country's budget watchdog said on Thursday.


"Fiscal sustainability requires that government debt cannot ultimately grow faster than the economy," Kevin Page, the parliamentary budget officer, said in a report that looked at likely trends over the next 75 years.


Page's team projected government debt relative to the size of the economy over the long term in the light of current spending and taxes as well as projected demographic and economic trends.


"(Our) debt-to-GDP projection indicates that the current federal and provincial-territorial fiscal structure is not sustainable over the long term," he wrote.


"Addressing this fiscal gap and restoring sustainability to public finances would require permanent policy actions of 2.7 percent of gross domestic product, either to raise taxes, reduce overall program spending, or some combination of both."


Page said slower labor force growth caused by an aging population would reduce annual average real gross domestic product growth from the 2.6 percent observed over the 1977-2010 period to 1.8 percent over the 2011-2086 period.





David Ljunggren
Reuters
29 Sep, 2011

CONRAD BLACK: IN PRAISE OF THE ALPHA FEMALE

It is surely time to recognize what an immense improvement has been wrought in world standards of governance by the rise of female national leaders.

Golda Meir and Indira Gandhi were effectively the pioneers, among democratically elected leaders, though the genius of a considerable number of previous empresses and queens gave a foretaste of what the world was denying itself in excluding women from its highest public offices (and most other important positions). Queen Elizabeth I was the greatest British monarch; Victoria was certainly competent, and although such comparisons are odious, the present queen surely has better judgment than have most of the 12 British and 11 Canadian prime ministers who have served her.


Meir was a strong Israeli foreign minister and a tough prime minister, who though somewhat taken by surprise in the Yom Kippur war in 1973, made Israel a nuclear power and governed intelligently in the socialistic tradition. Indira Gandhi, after a brief interregnum following the death of her father, Jawaharlal Nehru, inherited the leadership of her country. She, too, continued the socialistic policies of her father and (unfortunately) the insufferably pretentious practice of sitting in the rose garden of the prime minister’s residence in New Delhi, fondling a flower and explaining that India was the moral arbiter of the world because of its secular spirituality and ethical exaltedness, despite its poverty, indulged primitiveness, hypocrisy and corruption.


She sliced Pakistan in two in 1971, created Bangladesh (greeted on the day of its founding by Henry Kissinger as “a basket case, but not our basket case”); promoted India’s nuclear program, but suffered temporary electoral defeat over her imposition of mandatory sterilization to combat rampant population growth (impossible in even a quasi-democracy). South Asia was a rough-and-tumble political environment, as Indira was assassinated, as were her protegé Mujibar Rahmin, the George Washington of Bangladesh; Indira’s son Rajiv, who succeeded her; and the next leaders of Pakistan — Zulfiqar Ali Bhutto, General Zia ul-Haq, and Bhutto’s daughter Benazir. Indira Gandhi’s and Benazir Bhutto’s perseverance in such an environment was remarkable, and their heirs rule their countries yet.

The real breakthrough in leadership by women in sophisticated democracies came with Margaret Thatcher in the United Kingdom. She had been education secretary in the government of Edward Heath (1970-4), who essentially continued what was called “Butskillism” (after Conservative deputy leader Rab Butler and 1950s-era Labour Party leader Hugh Gaitskell), whereby there were only marginal social policy differences between the two main parties, whatever rhetorical excesses they engaged in at each others’ expense at election time.


Thatcher opposed Heath at the party convention of 1975, when senior alternatives declined to bell the cat. She was elected leader of the opposition, slogged through four years in that role, and only narrowly won the 1979 general election over James Callaghan, although Britain was being closely monitored by the IMF; sluggish economic growth and a labour movement that shut down anything on a whim of local officials and was completely out of control. The metropolitan London garbage collectors and even the undertakers were on strike in the winter of early 1979, while London suffered extensive electricity brown-outs two and three days a week because of union industrial action. Despite the lamentable state of the country, Thatcher’s victory was narrow because of her radical program and, it is conjectured, because she was the country’s first major-party female leader. When she was elected leader, and went to the Conservative Party’s informal social headquarters, the Carlton Club, she was told that ladies were not allowed, other than as guests. “They are now,” she famously said as she sailed majestically past the concierge.


Margaret Thatcher cut personal income tax rates to 40%, required secret ballots for strike authorizations, reoriented the British work force to more modern industries by reducing or ending subsidies and massive privatization (including virtually giving public housing to its occupants), generated exhilarating economic growth, and expelled the Argentineans from the Falkland Islands, which they had illegally seized. (This produced the ancillary benefit of toppling the military junta in that country, and relaunching Argentinean democracy, which has recently re-elected a female president.)

She became the first prime minister in British history since the radical expansion of the franchise in the First Reform Act of 1832 to win three consecutive full terms. (Tony Blair has replicated the feat, but his party, unlike Thatcher’s, was defeated at the next election.) She is generally reckoned to be surpassed only by Winston Churchill as the greatest British prime minister of the 20th century. (Disclosure: she was my sponsor as a member of the House of Lords, and proposed a toast to Barbara and me at our wedding dinner, and I have been a vociferous supporter of hers since I arrived commercially in the U.K. 25 years ago.)


Now the world is festooned with prominent women heads of government and public leaders. Angela Merkel, the seventh federal German chancellor, is the East German daughter of a Protestant clergyman; she has none of the glamour of Indira Gandhi and not much of the panache of Margaret Thatcher, but is an effective leader at a time when Germany is benignly returning to the role it held under Bismarck as Europe’s most important power. France held that honour in the ’20s and ’90s and the Soviet Union did from 1945 until its dissolution.


Chancellor Merkel is rightly refusing to follow the American example of having the European Central bank (largely in fact a German bank in terms of the backing of its reserves) buy the Eurobonds of distressed countries or approve the issuance of Eurobonds whose proceeds would be funnelled to the needs of those countries. The fallout will be severe, but Germany will protect the integrity of the euro and enjoy the irony of many neighbouring countries who in living memory have fought the oppression and overlordship of Germany with desperate bravery, beseeching German economic suzerainty. (The welcome possibility has recently emerged that Merkel might approve such bond issues in the event of sovereign default and imposed programs of market liberalization and not just self-amplifying Scroogian austerity.)


Turning back to India, Sonia Gandhi, Indira Gandhi’s daughter-in-law, an Italian and a Roman Catholic, has guided the Congress Party from the socialistic economic wasteland of its founders and their early continuators, to a policy of deregulation, incentivization of economic growth, and the systematic reduction of poverty and growth of the middle class that has made Indian economic progress roughly parallel to China’s (though many corrupt government practices remain). It has achieved this while retaining a plausible democracy such as has never existed in China (other than in Taiwan in recent decades). She had the wisdom not to accept the proffered leadership of Congress, but to be party chairman, and is preparing to deliver the leadership, for the fourth consecutive generation, to her son. Sonia Gandhi is one of the world’s greatest and most under-recognized political strategists, in one of the world’s most important countries.


The Burmese democratic leader, Aung San Suu Kyi has been well-lionized in the West and awarded the Nobel Peace Prize, but few could appreciate the courage required to lead resistance through four years of imprisonment and approximately a decade of house arrest at the hands of one of the most obtusely despotic regimes in the world. She, aided by the greed and overbearing presumption of the Chinese, has brought the Burmese leaders to a stage of partial democratization, in order to facilitate a resumption of functioning relations with the West. This could be a democratic victory as important as, and comparably heroic to, that of Nelson Mandela, by a thoroughly westernized, very brave woman in an insular hermit country.


Thailand’s prime minister, Yingluck Shinawatra, is standing in for her brother, who made a huge fortune in computer software and cracked the national political monopoly of the generals, courtiers, and Bangkok rich by founding a rural-based people’s party (“Thais Loving Thais”), took office, began to liberalize (while abolishing capital gains taxes and selling his business at a huge profit), until he was sent packing by the military. Thailand has been laid low by floods, and the Prime Minister, wading through knee-deep water (in exiguous shorts), is fighting for her family’s position. She bears watching.


Finally, the Ukraine’s Julia Timoshenko is a beautiful woman and a fiery orator whose political and financial ethics are not above suspicion. She is now a political prisoner, on trumped-up charges, having narrowly lost the last presidential election. She has attracted the solicitude of Europe and much of the world, and claims to be suffering from a mysterious illness (so mysterious it has no known or identifiable symptoms, as she exhibits none). She is a Slavic Evita, with a non-political husband, and she will be back.


In light of all this, it is worth reflecting, generally, on what the world was missing, with a 50%-restricted talent pool, in competition for high public office, prior to about 1965. And one welcomes the day when a similar transformation occurs in those parts of the world — the Arab Middle East and sub-Saharan Africa, in particular — that so far have suppressed their own Ghandis, Thatchers, Meirs and Timoshenkos.

Conrad Black 

National Post
Dec 3, 2011

ARE YOU READY FOR THE ERA OF "BIG DATA"?

Radical customization, constant experimentation, and novel business models will be new hallmarks of competition as companies capture and analyze huge volumes of data. Here’s what you should know.


The top marketing executive at a sizable US retailer recently found herself perplexed by the sales reports she was getting. A major competitor was steadily gaining market share across a range of profitable segments. Despite a counterpunch that combined online promotions with merchandizing improvements, her company kept losing ground.


When the executive convened a group of senior leaders to dig into the competitor’s practices, they found that the challenge ran deeper than they had imagined. The competitor had made massive investments in its ability to collect, integrate, and analyze data from each store and every sales unit and had used this ability to run myriad real-world experiments. At the same time, it had linked this information to suppliers’ databases, making it possible to adjust prices in real time, to reorder hot-selling items automatically, and to shift items from store to store easily. By constantly testing, bundling, synthesizing, and making information instantly available across the organization—from the store floor to the CFO’s office—the rival company had become a different, far nimbler type of business.


What this executive team had witnessed first hand was the game-changing effects of big data. Of course, data characterized the information age from the start. It underpins processes that manage employees; it helps to track purchases and sales; and it offers clues about how customers will behave.


But over the last few years, the volume of data has exploded. In 15 of the US economy’s 17 sectors, companies with more than 1,000 employees store, on average, over 235 terabytes of data—more data than is contained in the US Library of Congress. Reams of data still flow from financial transactions and customer interactions but also cascade in at unparalleled rates from new devices and multiple points along the value chain. Just think about what could be happening at your own company right now: sensors embedded in process machinery may be collecting operations data, while marketers scan social media or use location data from smartphones to understand teens’ buying quirks. Data exchanges may be networking your supply chain partners, and employees could be swapping best practices on corporate wikis.


All of this new information is laden with implications for leaders and their enterprises.1 Emerging academic research suggests that companies that use data and business analytics to guide decision making are more productive and experience higher returns on equity than competitors that don’t.2 That’s consistent with research we’ve conducted showing that “networked organizations” can gain an edge by opening information conduits internally and by engaging customers and suppliers strategically through Web-based exchanges of information.3


Over time, we believe big data may well become a new type of corporate asset that will cut across business units and function much as a powerful brand does, representing a key basis for competition. If that’s right, companies need to start thinking in earnest about whether they are organized to exploit big data’s potential and to manage the threats it can pose. Success will demand not only new skills but also new perspectives on how the era of big data could evolve—the widening circle of management practices it may affect and the foundation it represents for new, potentially disruptive business models.


Five big questions about big data


In the remainder of this article, we outline important ways big data could change competition: by transforming processes, altering corporate ecosystems, and facilitating innovation. We’ve organized the discussion around five questions we think all senior executives should be asking themselves today.


At the outset, we’ll acknowledge that these are still early days for big data, which is evolving as a business concept in tandem with the underlying technologies. Nonetheless, we can identify big data’s key elements. First, companies can now collect data across business units and, increasingly, even from partners and customers (some of this is truly big, some more granular and complex). Second, a flexible infrastructure can integrate information and scale up effectively to meet the surge. Finally, experiments, algorithms, and analytics can make sense of all this information. We also can identify organizations that are making data a core element of strategy. In the discussion that follows and elsewhere in this issue, we have assembled case studies of early movers in the big data realm (see “Seizing the potential of ‘big data’” and the accompanying sidebar, “AstraZeneca’s ‘big data’ partnership,” on mckinseyquarterly.com.)


Even as big data changes the game for virtually every sector, it also tilts the playing field, favoring some companies and industries, particularly in the early stages of adoption. To understand those dynamics, we examined 20 sectors in the US economy, sized their contributions to GDP, and developed two indexes that estimate each sector’s potential for value creation using big data, as well as the ease of capturing that value.1


As the accompanying sector map shows (exhibit), financial players get the highest marks for value creation opportunities. Many of these companies have invested deeply in IT and have large data pools to exploit. Information industries, not surprisingly, are also in this league. They are data intensive by nature, and they use that data innovatively to compete by adopting sophisticated analytic techniques.


The public sector is the most fertile terrain for change. Governments collect huge amounts of data, transact business with millions of citizens, and, more often than not, suffer from highly variable performance. While potential benefits are large, governments face steep barriers to making use of this trove: few managers are pushed to exploit the data they have, and government departments often keep data in siloes.


Fragmented industry structures complicate the value creation potential of sectors such as health care, manufacturing, and retailing. The average company in them is relatively small and can access only limited amounts of data. Larger players, however, usually swim in bigger pools of data, which they can more readily use to create value.


The US health care sector, for example, is dotted by many small companies and individual physicians’ practices. Large hospital chains, national insurers, and drug manufacturers, by contrast, stand to gain substantially through the pooling and more effective analysis of data. We expect this trend to intensify with changing regulatory and market conditions. In manufacturing, too, larger companies with access to much internal and market data will be able to mine new reservoirs of value. Smaller players are likely to benefit only if they discover innovative ways to share data or grow through industry consolidation. The same goes for retailing, where—despite a healthy strata of data-rich chains and big-box stores on the cutting edge of big data—most players are smaller, local businesses with a limited ability to gather and analyze information.


A final note: this analysis is a snapshot in time for one large country. As companies and organizations sharpen their data skills, even low-ranking sectors (by our gauges of value potential and data capture), such as construction and education, could see their fortunes change.


Notes

The big data value potential index takes into account a sector’s competitive conditions, such as market turbulence and performance variability; structural factors, such as transaction intensity and the number of potential customers and business partners; and the quantity of data available. The ease-of-capture index takes stock of the number of employees with deep analytical talent in an industry, baseline investments in IT, the accessibility of data sources, and the degree to which managers make data-driven decisions.


In addition, we’d suggest that executives look to history for clues about what’s coming next. Earlier waves of technology adoption, for example, show that productivity surges not only because companies adopt new technologies but also, more critically, because they can adapt their management practices and change their organizations to maximize the potential. We examined the possible impact of big data across a number of industries and found that while it will be important in every sector and function, some industries will realize benefits sooner because they are more ready to capitalize on data or have strong market incentives to do so (see sidebar, “Parsing the benefits: Not all industries are created equal”).


The era of big data also could yield new management principles. In the early days of professionalized corporate management, leaders discovered that minimum efficient scale was a key determinant of competitive success. Likewise, future competitive benefits may accrue to companies that can not only capture more and better data but also use that data effectively at scale. We hope that by reflecting on such issues and the five questions that follow, executives will be better able to recognize how big data could upend assumptions behind their strategies, as well as the speed and scope of the change that’s now under way.

1. What happens in a world of radical transparency, with data widely available?


As information becomes more readily accessible across sectors, it can threaten companies that have relied on proprietary data as a competitive asset. The real-estate industry, for example, trades on information asymmetries such as privileged access to transaction data and tightly held knowledge of the bid and ask behavior of buyers. Both require significant expense and effort to acquire. In recent years, however, online specialists in real-estate data and analytics have started to bypass agents, permitting buyers and sellers to exchange perspectives on the value of properties and creating parallel sources for real-estate data.


Beyond real estate, cost and pricing data are becoming more accessible across a spectrum of industries. Another swipe at proprietary information is the assembly by some companies of readily available satellite imagery that, when processed and analyzed, contains clues about competitors’ physical facilities. These satellite sleuths glean insights into expansion plans or business constraints as revealed by facility capacity, shipping movements, and the like.


One big challenge is the fact that the mountains of data many companies are amassing often lurk in departmental “silos,” such as R&D, engineering, manufacturing, or service operations—impeding timely exploitation. Information hoarding within business units also can be a problem: many financial institutions, for example, suffer from their own failure to share data among diverse lines of business, such as financial markets, money management, and lending. Often, that prevents these companies from forming a coherent view of individual customers or understanding links among financial markets.


Some manufacturers are attempting to pry open these departmental enclaves: they are integrating data from multiple systems, inviting collaboration among formerly walled-off functional units, and even seeking information from external suppliers and customers to cocreate products. In advanced-manufacturing sectors such as automotive, for example, suppliers from around the world make thousands of components. More integrated data platforms now allow companies and their supply chain partners to collaborate during the design phase—a crucial determinant of final manufacturing costs.


2. If you could test all of your decisions, how would that change the way you compete?


Big data ushers in the possibility of a fundamentally different type of decision making. Using controlled experiments, companies can test hypotheses and analyze results to guide investment decisions and operational changes. In effect, experimentation can help managers distinguish causation from mere correlation, thus reducing the variability of outcomes while improving financial and product performance.

Robust experimentation can take many forms. Leading online companies, for example, are continuous testers. In some cases, they allocate a set portion of their Web page views to conduct experiments that reveal what factors drive higher user engagement or promote sales. Companies selling physical goods also use experiments to aid decisions, but big data can push this approach to a new level. McDonald’s, for example, has equipped some stores with devices that gather operational data as they track customer interactions, traffic in stores, and ordering patterns. Researchers can model the impact of variations in menus, restaurant designs, and training, among other things, on productivity and sales.


Where such controlled experiments aren’t feasible, companies can use “natural” experiments to identify the sources of variability in performance. One government organization, for instance, collected data on multiple groups of employees doing similar work at different sites. Simply making the data available spurred lagging workers to improve their performance.


Leading retailers, meanwhile, are monitoring the in-store movements of customers, as well as how they interact with products. These retailers combine such rich data feeds with transaction records and conduct experiments to guide choices about which products to carry, where to place them, and how and when to adjust prices. Methods such as these helped one leading retailer to reduce the number of items it stocked by 17 percent, while raising the mix of higher-margin private-label goods—with no loss of market share.


3. How would your business change if you used big data for widespread, real-time customization?


Customer-facing companies have long used data to segment and target customers. Big data permits a major step beyond what until recently was considered state of the art, by making real-time personalization possible. A next-generation retailer will be able to track the behavior of individual customers from Internet click streams, update their preferences, and model their likely behavior in real time. They will then be able to recognize when customers are nearing a purchase decision and nudge the transaction to completion by bundling preferred products, offered with reward program savings. This real-time targeting, which would also leverage data from the retailer’s multitier membership rewards program, will increase purchases of higher-margin products by its most valuable customers.


Retailing is an obvious place for data-driven customization because the volume and quality of data available from Internet purchases, social-network conversations, and, more recently, location-specific smartphone interactions have mushroomed. But other sectors, too, can benefit from new applications of data, along with the growing sophistication of analytical tools for dividing customers into more revealing microsegments.


One personal-line insurer, for example, tailors insurance policies for each customer, using fine-grained, constantly updated profiles of customer risk, changes in wealth, home asset value, and other data inputs. Utilities that harvest and analyze data on customer segments can markedly change patterns of power usage. Finally, HR departments that more finely segment employees by task and performance are beginning to change work conditions and implement incentives that improve both satisfaction and productivity.4


4. How can big data augment or even replace management?


Big data expands the operational space for algorithms and machine-mediated analysis. At some manufacturers, for example, algorithms analyze sensor data from production lines, creating self-regulating processes that cut waste, avoid costly (and sometimes dangerous) human interventions, and ultimately lift output. In advanced, “digital” oil fields, instruments constantly read data on wellhead conditions, pipelines, and mechanical systems. That information is analyzed by clusters of computers, which feed their results to real-time operations centers that adjust oil flows to optimize production and minimize downtimes. One major oil company has cut operating and staffing costs by 10 to 25 percent while increasing production by 5 percent.


Products ranging from copiers to jet engines can now generate data streams that track their usage. Manufacturers can analyze the incoming data and, in some cases, automatically remedy software glitches or dispatch service representatives for repairs. Some enterprise computer hardware vendors are gathering and analyzing such data to schedule preemptive repairs before failures disrupt customers’ operations. The data can also be used to implement product changes that prevent future problems or to provide customer use inputs that inform next-generation offerings.


Some retailers are also at the forefront of using automated big data analysis: they use “sentiment analysis” techniques to mine the huge streams of data now generated by consumers using various types of social media, gauge responses to new marketing campaigns in real time, and adjust strategies accordingly. Sometimes these methods cut weeks from the normal feedback and modification cycle.


But retailers aren’t alone. One global beverage company integrates daily weather forecast data from an outside partner into its demand and inventory-planning processes. By analyzing three data points—temperatures, rainfall levels, and the number of hours of sunshine on a given day—the company cut its inventory levels while improving its forecasting accuracy by about 5 percent in a key European market.


The bottom line is improved performance, better risk management, and the ability to unearth insights that would otherwise remain hidden. As the price of sensors, communications devices, and analytic software continues to fall, more and more companies will be joining this managerial revolution.


5. Could you create a new business model based on data?


Big data is spawning new categories of companies that embrace information-driven business models. Many of these businesses play intermediary roles in value chains where they find themselves generating valuable “exhaust data” produced by business transactions. One transport company, for example, recognized that in the course of doing business, it was collecting vast amounts of information on global product shipments. Sensing opportunity, it created a unit that sells the data to supplement business and economic forecasts.


Another global company learned so much from analyzing its own data as part of a manufacturing turnaround that it decided to create a business to do similar work for other firms. Now the company aggregates shop floor and supply chain data for a number of manufacturing customers and sells software tools to improve their performance. This service business now outperforms the company’s manufacturing one.


Big data also is turbocharging the ranks of data aggregators, which combine and analyze information from multiple sources to generate insights for clients. In health care, for example, a number of new entrants are integrating clinical, payment, public-health, and behavioral data to develop more robust illness profiles that help clients manage costs and improve treatments.


And with pricing data proliferating on the Web and elsewhere, entrepreneurs are offering price comparison services that automatically compile information across millions of products. Such comparisons can be a disruptive force from a retailer’s perspective but have created substantial value for consumers. Studies show that those who use the services save an average of 10 percent—a sizable shift in value.


Confronting complications


Up to this point, we have emphasized the strategic opportunities big data presents, but leaders must also consider a set of complications. Talent is one of them. In the United States alone, our research shows, the demand for people with the deep analytical skills in big data (including machine learning and advanced statistical analysis) could outstrip current projections of supply by 50 to 60 percent. By 2018, as many as 140,000 to 190,000 additional specialists may be required. Also needed: an additional 1.5 million managers and analysts with a sharp understanding of how big data can be applied. Companies must step up their recruitment and retention programs, while making substantial investments in the education and training of key data personnel.


The greater access to personal information that big data often demands will place a spotlight on another tension, between privacy and convenience. Our research, for example, shows that consumers capture a large part of the economic surplus that big data generates: lower prices, a better alignment of products with consumer needs, and lifestyle improvements that range from better health to more fluid social interactions.5 As a larger amount of data on the buying preferences, health, and finances of individuals is collected, however, privacy concerns will grow.


That’s true for data security as well. The trends we’ve described often go hand in hand with more open access to information, new devices for gathering it, and cloud computing to support big data’s weighty storage and analytical needs. The implication is that IT architectures will become more integrated and outward facing and will pose greater risks to data security and intellectual property. For some ideas on how leaders should respond, see “Meeting the cybersecurity challenge.”


Although corporate leaders will focus most of their attention on big data’s implications for their own organizations, the mosaic of company-level opportunities we have surveyed also has broader economic implications. In health care, government services, retailing, and manufacturing, our research suggests, big data could improve productivity by 0.5 to 1 percent annually. In these sectors globally, it could produce hundreds of billions of dollars and euros in new value.


In fact, big data may ultimately be a key factor in how nations, not just companies, compete and prosper. Certainly, these techniques offer glimmers of hope to a global economy struggling to find a path toward more rapid growth. Through investments and forward-looking policies, company leaders and their counterparts in government can capitalize on big data instead of being blindsided by it.


Source: McKinsey Global Institute October 2011

Brad Brown, Michael Chui, and James Manyika


About the Authors



Brad Brown is a director in McKinsey’s New York Office; Michael Chui is a senior fellow with the McKinsey Global Institute (MGI) and is based in the San Francisco office; James Manyika is a director of MGI and a director