Technology has led a dynamic transformation of the exchange industry, and cloud strategies are an important factor in unlocking the revenue potential of exchange data.  The following article about how to monetize exchange data via the cloud was recently published in “How To Build An Exchange,” a fascinating must-read collection from our friends at Mondo Visione.  Request access to the full collection from Mondo Visione here. 

_____________

Monetizing Market Data in an Evolving Global Marketplace

The global environment for exchanges and data originators is rapidly evolving with exploding data volumes, increasing and unpredictable regulation, expanding variety of financial instruments, fragmenting market, competitive encroachment and the need for new sources of revenue. Beyond these challenges, the race to microsecond latency will stretch the infrastructure capacities and budgets at major financial services firms.

Exchanges and data originators are struggling to service the opportunities of the future with the products and solutions of the past. A paradigm shift is rapidly approaching as old monolithic systems designed for universal applicability are being replaced by new custom tailored micro and mobile apps designed to satisfy the needs of a specific knowledge worker or user type.

The emergence of new mobile technologies and use-specific applications is altering the usage patterns of market data and changing the ways market data is purchased and consumed. If an emerging exchange is to succeed, it must tackle these issues in new ways.

A few years from now the market data application mix will boil down to two distinct flavors: low latency and everything else. How will exchanges capture new market data revenue profitably and manage the delivery of these two distinct flavors when their target applications, usage patterns and infrastructure requirements are so dramatically different?

Given its mission-critical impact on trading efficiency, low latency infrastructure has historically received the bulk of attention and investment. Meanwhile, legacy data distribution systems are crumbling under the weight of the increasing volumes of data produced by low-latency trading. The cost and complexity of servicing everything else is skyrocketing. Enter cloud computing. Once thought to be the domain of small and medium-sized businesses, cloud strategies are now seeing investment from exchanges and data originators to gain economies-of-scale, simplify the access and delivery of market data, decrease time-to-market for new products and add new revenue streams from new and underserviced segments.

In August of 2011, The Melbourne Group laid out the following cloud action plan for existing and emerging exchanges:

Make Core Market Data Assets Available from the Cloud

By creating an Internet-based cloud offering using standard cloud APIs, exchanges can offer both institutional and retail customers a means to access their core market data assets on-demand while driving down both the costs of access for the customer and the costs of service and maintenance for all.

Create Cross-Selling Opportunities with Direct Feed Customers

Direct feeds designed to service automated trading applications are poor vehicles for usercentric and mobile applications. A cloud offering coupled with an exchange’s direct feeds allows buy and sell-side firms to synchronize front and back-office operations seamlessly.

Monetize Exchange Data Assets by Targeting Unmet Business Needs

Without the cloud, the sheer volume of stale data sets make broad delivery cost-prohibitive to anyone but the largest clients. With the cloud, exchanges can move away from the one-size-fits-all approach and offer custom tailored products that allow customers to select and consume only the data required for their specific applications, modeling or back-testing.

Expand and Simplify Global Access and Delivery

Cloud-based delivery can be complemented with an e-commerce purchase and provisioning system to access new geographic regions and new customers through a 24×7 web presence which allows customers to select, purchase and use an exchange’s data products without a local sales organization, or any sales department intervention at all.

Increase Revenue by Expanding Retail Channels

A scalable, low-cost cloud platform allows exchanges to distribute exchange data directly to retail websites and investors. By accessing these channels directly, exchanges open revenue streams currently outside their reach and strengthen their brand within new and existing markets.

Deliver New Products Faster to Meet Emerging Customer Needs

New trading products create new information needs for financial services firms and market participants. The cloud allows exchanges to provide immediate global access to new market data products, removing constraints imposed by traditional market data vendors and eliminating infrastructure investments by customers.

Increase Market Transparency for OTC Derivatives and Complex Products

New exchange-traded products and new reporting requirements brought forth by Dodd-Frank, RegNMS and MiFid create new information needs for financial services firms and market participants. A cloud solution can provide both market participants and regulators easy global access to activity in these markets, providing accurate and auditable information that helps financial institutions manage risk and meet regulatory and internal compliance requirements.

The Last Hold-Out

The fixed income trading world has been the last hold-out in the broad shift towards electronic trading and more transparent centralized trading venues. We’ve already witnessed the other popular financial instruments including equities, options, futures, and currencies, succumb to the inevitable, but fixed income is still dominated by the murky business of OTC trading. There are signs, however, that even this asset class has finally reached a tipping-point, and that we are now on the verge of a new world where the buy-side’s demands for transparency will finally be realized.

To understand what this future may look like we must first understand how fixed income trading got to where it is today.

When Two Sides Collide

The fixed income trading world has always been defined by the opposing interests of the buy-side and the sell-side.

The buy-side has almost uniformly expressed dissatisfaction at the whole fixed income trading process. Complaints include a complete lack of transparency, an inability to obtain reliable price discovery and market data, the high-cost, and the general inefficiency when compared to the electronic trading of other asset classes. More recently there has been intense pressure to include fixed income in electronic multi-asset trading class strategies.

The sell-side on the other-hand has been highly motivated to keep things status quo. Over the years fixed income trading has been a cash cow business for the large Wall Street dealers. For example, fully 52% of Goldman Sachs’ $45 billion revenue in 2009 came from their fixed income trading division. The sell-side understands that moving fixed income trading away from the OTC model to more exchange-like platforms and Alternative Trading Systems (ATS’s) will almost certainly result in greatly reduced profits. The sell-side in their defense has argued that the sheer volume (3 million fixed-income securities and counting) and the inherent complexity of fixed income products does not lend itself to a more standardized and centralized market.

Innovation around the Edges

That is not to say that there has not been innovation or change. Over the years there have been some key developments that have allowed the fixed income trading world to creep towards a more electronic and transparent marketplace:

1998 – Tradeweb introduces an electronic request-for-quote-driven multi-dealer platform focused on U.S. treasury bonds.

2000 – MarketAxcess introduces an electronic request-for-quote-driven multi-dealer platform focused on corporate bonds.

2000 – FIX, the widely-used equity-focused electronic trading protocol, is expanded to include fixed income features.

2002 – NASD introduces TRACE (Trade Reporting and Compliance Engine) which captures and disseminates consolidated fixed income transactions from the secondary bond market.

On the retail side we’ve also seen some innovation with more fixed income market data becoming available on websites like “Investinginbonds.com” and “Yahoo! Bond Center”.

The Credit-Crisis and the Regulatory Response

The credit crisis and the yet-to-be-fully understood regulatory response has upset the status quo of the fixed income trading world and will result in the demands of the buy-side finally overwhelming the interests of the sell-side dealers.

The sell-side lost billions during the crisis and is already accepting that the writing is on the wall. In some cases they are not waiting for regulation and are in the process of downsizing their fixed income trading businesses. We are also seeing them less willing to risk their own capital in fixed income trading.

On the regulation front there have been a number of major initiatives including the Dodd-Frank Wall Street Reform and Consumer Protection Act and Europe’s MiFID II, both of which force transparency by mandating that swaps and other derivatives be traded on exchanges and Swap Execution Facilities (SEFs). The Volcker Rule will further strike at dealers by outlawing proprietary trading and tactics such as the highly profitable ‘flow trading’.

The Electronic and Transparent Future

Perhaps the biggest advantage of this new fixed income electronic trading world will be the broad availability of fixed income market data. As we’ve seen with other asset classes that have gone through this electronic transition the availability of accurate market data will result in the buy-side becoming much more focused on cost and efficiency. There will be increased interest in best-execution and over time we will see a more efficient marketplace.

Another key trend that we can expect is the ‘retailization’ of fixed income trading. As fixed income market data becomes ubiquitous we will see the retail investor want greater direct involvement in electronic fixed income trading. This will result in more services, websites, and even mobile apps aimed exclusively at the retail bond investor.

There is little doubt that we are in the midst of a technological sea change as the world moves from an Internet that was tethered to PCs, to a world where the Internet can be accessed from anywhere, through a wide array of always-on smartphones and tablets. We’ve all heard the impressive statistics that support this trend with just over 400 million smartphones expected to be sold globally in 2011, and Gartner forecasting that the mobile web will grow to 1 billion smartphones and 320 million tablets sold in 2015.

A Consumer-Driven Revolution

To date this has been very much a consumer-driven revolution with the vast majority of Apple’s 1 billion monthly app downloads being aimed at the consumer. This, however, is beginning to change with more and more consumers demanding the convenience of mobile devices and mobile apps inside the enterprise. We are now seeing growing enterprise adoption with Apple reporting just this week that 92% of Fortune 500 companies are either testing or deploying the iPad. This is an astounding number especially since the iPad product itself did not exist 2 years ago. Closer to home, according to Good Technology, a firm that manages mobile devices for large companies, financial services firms accounted for almost half of all new iPad activations in the second quarter of 2011.

So as this mobile app revolution inexorably makes its way to the buy-side there are a number of questions to be answered:  How will buy-side mobile apps affect the technology landscape? How will buy-side mobile apps change the way people do their jobs? How will buy-side mobile apps allow firms to better serve their clients?

Monolithic Management Systems Meet Buy-Side Mobile Apps

Today, when we look at the applications used by the buy-side we can see that it is still dominated by the same trading, order management, and portfolio management systems, that have been around for the last 20 years. It could be argued that not much has changed. Buy-side employees are still spending most of their day working with these large monolithic management systems. Of course, there have been advances particularly in the areas of trading, workflow, integration, and Software-as-a-Service, but the components that make up the software infrastructure of the buy-side have essentially remained the same.

We believe that the arrival of mobile apps to the buy-side will have a profound  impact on the buy-side’s technology, employees, and ultimately the way clients are serviced. This revolution is different than other over-hyped technology fads because it is being driven by the end-user rather than by some easily-ignored top-down management dictum. Buy-side employees have already experienced the power of mobile apps as consumers and will begin using them at work, whether they are approved by their IT department or not.

We see the impact of buy-side mobile apps evolving in three distinct phases:

Phase 1 – View-Only Buy-Side Mobile Apps

Most of today’s buy-side mobile apps build on the anywhere capability of mobile devices and are strictly view-only type offerings, that display the same data that users see on their trading, order management, and portfolio management systems. These research, P&L, and client information apps augment the existing monolithic management systems and are many times used by employees outside of work during commutes or client visits.  The apps are  frequently designed to look like dashboards with the ability to drill-down to more detailed information. There are many examples out there but some of the most widely used are Bloomberg Anywhere for iPad, Merlin’s Compass for iPad, and Fidelity’s WealthCentral  Android/iPhone Mobile Apps.

Phase 2 – Action-Oriented Buy-Side Mobile Apps

This phase goes beyond the view-only capabilities of Phase 1 and allows the user to act on the information presented by their mobile app. Users for example may be alerted to a trading opportunity or a back-office reconciliation issue that they can then immediately act-on, just as if they were sitting in front of their office PC. Again this phase really just serves to augment or duplicate the existing monolithic management system functionality.  Many of the above-named firms are now in the process of releasing these newer more action-oriented Phase 2 buy-side mobile apps.

Phase 3 – Proliferation of Buy-side Mobile Apps/Decomposition of Monolithic Management Systems

The final phase is when mobile apps will really come into their own and we will see them begin to replace rather than just augment pieces of monolithic management systems.  This phase will produce buy-side mobile apps that combine the anywhere/view-only nature of Phase 1, the action-oriented ability of Phase 2, with other unique qualities that only mobile apps can offer. A hint of this future is the StockTouch app (www.stocktouch.com), that is powered by Xignite market data. StockTouch uses the touchscreen interface of the iPhone and iPad to bring to life a visualization of the stock market. Their mobile app offers functionality that no traditional buy-side application can emulate.

Over time we will see a proliferation of these smaller buy-side mobile apps that focus in on specific functionality.  This will lead to a fragmented app market just like in the consumer world. As users spend more and more of their time using these mobile buy-side apps we will see the monolithic management systems become less relevant, ultimately leading to the their decomposition.

The last few years have been undeniably tough for the once brash hedge fund industry. Recent headlines do not suggest any improvement with August being the worst month for hedge funds since October 2008, and marquee firms like Paulson & Company firm down 34% year-to-date.  Prior to the crisis of 2008, the industry appeared to be on a steady upward trajectory, evolving from a small, scrappy upstart, that catered to high net worth investors, to a more formalized $2 trillion industry, that serviced the largest pension funds in the world. Since the crisis, however, the industry seems to have lost its way. What exactly happened and how can what we term the “Hedge Fund Cloud” return the industry to its former glory?

Institutionalize or Die

Pre-crisis, managers believed that the measure of success was not only returns but assets under management. In their race to acquire new assets, managers were motivated to “institutionalize” their infrastructure so that they could go after the really big allocations from large pension funds and endowments. For many firms this institutionalization meant leaving the relative simplicity of their single prime relationship to the much more complex world of building out their own multi-prime infrastructure. Almost overnight managers found themselves running complex and unwieldy businesses. Seemingly simple operations like adding a strategy, that required a new asset-class, or producing a new report, became long and involved IT projects.  Any thought of outsourcing any of this burden was dismissed because of perceived privacy and control concerns.

Prisoners of their own Hedge fund Infrastructure

The actual crisis further exposed the inflexibility of hedge funds’ infrastructures. Managers struggled to view their true exposure across asset classes and multi-prime relationships. Just when managers most needed their former agility they discovered that they had become prisoners of their own expensive infrastructures.

Fast forward to today. We are still experiencing the after effects of the crisis. A strong regulatory backlash response has been unavoidable. There is still tremendous uncertainly about the true impact of these new regulations, but what is certain, is that the business of running a hedge fund will become even more complex and costly.

How can the industry remove itself from this funk and prepare itself for the next crisis? The answer is that the industry needs to return to basics by once again making alpha generation its sole focus. The industry needs to regain its former investment agility. In short, managers need to get out of the running-a-hedge-fund business and get back to the investment business.

The Hedge Fund Cloud to the Rescue

Fortunately, the Hedge Fund Cloud offers managers the opportunity to get back to basics.  The Hedge Fund Cloud allows firms to focus on alpha generation by moving all non-core infrastructure to the cloud. The Hedge Fund Cloud is made up of the fast-maturing ecosystem of cloud-based hedge fund service providers who have now begun to offer institutional-grade infrastructure.

The three main components of the Hedge Fund Cloud are as follows:

1 – Hedge Fund Cloud – Software-as-a-Service (SaaS)

SaaS means that software can now be accessed through a thin client or browser, from anywhere, as a service. SaaS includes upgrades, disaster recovery, and is paid for on a subscription model. Many of the important hedge fund systems including execution management, order management, risk management, portfolio management are now available from the leading providers on a Software-as-a-Service basis.

2 – Hedge Fund Cloud – Data-as-a-Service (DaaS)

DaaS delivers financial market and reference data via web services or API’s rather than the traditional data feeds and flat files.  This approach allows firms the ability to query and use just specific subsets of data rather than downloading and storing masses of data. It also means that firms do not have to build out data management infrastructures.

3 – Hedge Fund Cloud – Infrastructure-as-a-Service (IaaS)

IaaS involves the outsourcing of physical hardware such as telephony, storage, servers, and networking components.

To take advantage of this new model, managers must now take an impartial eye to their existing hedge fund infrastructures and ask themselves what really is vital to alpha generation. Everything else should be stripped away, moved to the cloud, and essentially turned into a utility. This return to basics offers the industry the best chance to get back to the pre-crisis glory days and to restore its mojo.

Perhaps nowhere more in the diverse world of hedge fund strategies is the prospect of alpha decay more unsettling than at high-frequency trading firms. In many ways high-frequency trading firms are now facing a reality that other hedge funds with more esoteric strategies may one day face – too much money chasing a finite amount of alpha.

The End of the Hardware Arms Race

The latest figures indicate that high-frequency trading now accounts for somewhere between 60-70% of trading volume in the US. This is up from around 35% just five short years ago. High-frequency trading firms, that in the past have been among the most profitable on Wall Street, are now seeing that increased competition has crowded out many of their traditional strategies. High-frequency trading firms have responded by co-locating their black boxes and by throwing ever more expensive hardware at the problem. This approach has worked for some of the larger, better-funded firms, but only postpones the inevitable.

What will happen when this hardware arms race meets the laws of physics? The answer is that many of these firms will need to develop high-frequency trading strategies where alpha is still relatively abundant and competition less fierce.

New High-Frequency Trading Alpha Opportunities

The two most common opportunities that high-frequency trading are now exploring are as follows:

1. Alpha outside the US – The US was the first to develop electronic trading, other regions are still in the process of building out their electronic trading infrastructure. A recent Credit Suisse study estimated that high-frequency trading activity accounts for 35% in Europe and only 10% in Asia (excluding Japan). Many high-frequency trading firms are now seeking to port the strategies that worked so well in the US over to these less developed markets.
2. Multi-Dimensional Strategies – The traditional strategies of high-frequency trading have typically been one-dimensional involving the high-frequency trading in and out of a single large, liquid name. The newer high-frequency trading strategies are much more complex and multi-dimensional in nature searching for arbitrage opportunities across asset-classes, geographies etc.

The New Arms Race and Cloud-based Market Data

The common thread that runs through these new strategies is the need for access to more diverse and dispersed market data sets. Market data has always been the lifeblood of high-frequency trading firms. It has also been one of the real pain points of this industry with the requirement to build out large infrastructures to manage and query terabytes of information. This new world of alpha discovery means that high-frequency trading must now potentially analyze vastly greater amounts of data to discover opportunities. Furthermore, the competition to discover these new opportunities will be intense. In this new world alpha will also be more fleeting as firms quickly clone successful strategies. The new high-frequency trading arms race will be all about alpha discovery and time to implementation.

As the high-frequency trading industry’s search for alpha switches from a hardware focus, to a discovery/speed of implementation focus, it is clear that firms with access to large and diverse data sets, that are easy to query and work with, will be at a distinct competitive advantage. The traditional method of downloading vast data sets to a firm’s own data center will not work because of the cost and time delays involved. The only solution that is flexible enough to assist in such rapid discovery and implementation is a Cloud-based approach, where small sub-sets of global multi-asset class market data can be quickly isolated and analyzed. Xignite is the only firm committed to this vision of  large, diverse sets of Cloud-based data, all made accessible by our easy-to-use web services, that require no infrastructure, and that can implemented in a matter of minutes.

As this new arms race intensifies and the search for alpha becomes increasingly more desperate we will see more and more high-frequency trading firms abandon their in-house data management infrastructures for the flexibility and breath of data that Xignite’s Cloud-based solution offers.

New to Xignite?  Curious about what we do and what were all about?  This interview with Xignite’s Joel York may provide the answers your looking for.  Visit TMCnet.com to view other videos from leading Silicon Valley companies.

Algorithmic trading firms were put on notice last Monday when Commissioner Bart Chilton of the Commodity Futures Trading Commission (CFTC) said that high frequency trading (HFT) and algorithmic trading firms should be made legally responsible for maintaining a minimum set of testing and monitoring standards to prevent future flash crashes.

The possibility that algorithmic trading firms may need to support new regulatory requirements highlights the rapidly growing importance of the market data cloud and new cloud services like NASDAQ Data-On-Demand for back testing purposes.

In a bid to resuscitate the heroes of 1980’s sitcoms, Mr. Chilton declared, “I want to be like MacGyver. Remember, he was always trying to prevent crime before it happened.” So if you’re a criminally culpable algorithmic trading firm, you’d best beware—especially if Mr. Chilton gets his hands on a paper clip and a stick of chewing gum. Household items aside, Mr. Chilton also plans to rely on tools such as a “kind of Good Housekeeping Seal of Approval.” He’d like algorithmic trading systems to be tested by either exchanges or regulators before going live. And after going live, he’d like algorithmic trading systems to be monitored on an on-going basis.

So, if you’re engaged in algorithmic trading, you’ll probably want to make sure you’ve got systems in place to test your algorithms so you’re prepared for any regulatory requirements that might come down the pike. That’s where the market data cloud comes in. If you want to future proof your algorithmic trading systems against any possible testing requirements, you’ll want to look into how you can use the market data cloud to back test your algorithms with minimal expense, time and effort put into building new technology infrastructure.

On-demand market data cloud services like NASDAQ Data-On-Demand offer a new way to hold down both the cost and time investments of getting historical tick data to back test algorithmic trading systems. You can get only the slice of data you need for your specific test conditions, and you only pay for what you use, not the entire universe of data, which dramatically reduces your data costs. On-demand cloud services also let you get highly customized data sets in a variety of common formats (XML, CSV, JSON and so on) which reduces your need to build new technology infrastructure just to load the data into your algorithmic trading systems.

So if Mr. Chilton knocks at your door with the latest issue of Good Housekeeping under one arm, never fear. Just check out NASDAQ Data-On-Demand and rest assured you can handle any algorithmic trading regulatory requirements for back testing that he might have in mind without breaking the bank on new technology infrastructure.

(For more details on Mr. Chilton’s public statements, see the recent Securities Industry News coverage.)

This week NASDAQ was the first major exchange to launch an on-demand service offering Level 1 historical stock tick data.  As the first truly on-demand service for historical stock tick data, NASDAQ Data-On-Demand is leading a revolution that’s democratizing access to market data and reducing costs by orders of magnitude.

NASDAQ Data-On-Demand has made it easy and affordable to get access to historical stock tick data for everyone from individuals to large institutions.  Individuals can analyze historical stock tick data in Excel, while large institutions can develop large-scale applications like algorithmic trading applications, using NASDAQ Data-On-Demand’s historical stock tick data for back-testing their algorithms.

Individual investors, professional traders, analysts and other individual users can get started for as little as $10 per month and gain instant access to start pulling historical stock tick data into Excel directly from the NASDAQ Data-On-Demand website.  Individuals get the same sophisticated analytics and detailed Level 1 historical stock tick data previously available only to large institutions.  And most importantly, traders can cherry-pick specific data with pinpoint accuracy, eliminating the need for custom-developed systems to manage and process the data for them.  Since pricing is usage based, individuals can use highly targeted queries to pay only for the data they need, making historical stock tick data affordable even for individual investors.

Application developers can power their applications with historical stock tick data accessed in real time through web service APIs.  Gone are the days of loading terabytes of historical stock tick data into a giant database, hiring a DBA to optimize the database and building a data access layer to run queries that retrieve the data your applications need.  Now your applications can call an API directly from NASDAQ Data-On-Demand web services, pass parameters defining a specific subset of historical stock tick data, and get back only the tick data your application needs right at that moment.  No need for databases, servers or DBAs.  (Sorry, DBAs.)  And with no need for technology overhead, the costs and timelines for pulling data into applications plummet.

With NASDAQ taking up the on-demand banner and leading the charge with historical stock tick data, it will be interesting to see how this new development affects the on-going movement toward an on-demand market data cloud.

If you look at our new corporate bond master service, XigniteBondMaster, you’ll quickly see that you can use it to get information for over 125,000 U.S. corporate bonds, including issuer information, maturity dates, coupon rates and so on.  But you’re not likely to see one of the best features of the service until you roll up your sleeves and start digging around in the data:  the ability to look up official CUSIP identifiers with remarkable ease.

CUSIP identifiers are highly valued throughout the industry as the de facto standard for uniquely identifying over 9.1 million financial instruments.  And now you can easily look up CUSIPs for thousands of corporate bonds using XigniteBondMaster.  Interested in a particular issue, but not sure what its CUSIP is?  Try looking it up by the bond issuer!  Or just look it up by bond name.

All CUSIP data comes from CUSIP Global Services, an organization managed by Standard & Poor’s on behalf of the American Bankers Association specifically to oversee the official set of CUSIP identifiers.

You do need to have an agreement in place with CUSIP Global Services in order to receive CUSIP data, since this is highly valued data that must be licensed first.  But once you have an agreement in place, CUSIPs will automatically start appearing in your XigniteBondMaster outputs.

Market data providers and IT professionals have tough jobs. Every day financial markets spew out huge fountains of data that must be captured, routed, scrubbed, reconciled, stored and redistributed with dizzying speed and accuracy. The diversity of data is staggering, from low-latency pricing data for algorithmic trading to intermittent corporate actions such as stock splits, and from globally dispersed real-time currency exchange rates to aggregated end-of-day VWAP and NAV calculations. Optimizing and tuning the market data systems that keep this crucial information flowing smoothly and cost effectively is no easy task. What, if anything, can cloud computing offer to ease the challenge?

This is the second post in a series called “Thinking Out Cloud” with the aim of helping financial services and market data IT professionals charged with developing cloud computing strategies separate the cloud buzz from the cloud reality. This post explores the types of market data that naturally lend themselves to cloud computing (and those that do not) in order to identify the market data sweet spot for cloud computing.

First, it’s important to recognize that it is not the market data that is being outsourced to the cloud, but the on-premise market data management infrastructure. Therefore, the market data sweet spot for cloud computing is the intersection of data management systems that offer little competitive advantage, but are costly and difficult to maintain in-house. Both relative competitive advantage and relative level of difficulty will vary from firm to firm by business focus and IT capability respectively; however, there are aspects of the market data itself that can contribute significantly to the cost and complexity of maintaining an in-house data management system.

Hard to Use Market Data

If the market data comes in original feed formats that are not well suited to the particular use of the data in the final application, then considerable effort must be expended to make the market data application-ready. For example, a real-time streaming exchange feed is great for creating a stock ticker, but not so great when the goal is to analyze historical tick data or simply get an ad-hoc real-time quote for a single symbol, then there can be lots of programming involved to parse the feed, store the data, continuously refresh the database and create a data access layer that applications can easily utilize. Cloud computing is built on Web services that allow for multiple interfaces to the market data, so it is especially good at tailoring the data format to the specific needs of the application on the fly. For example, a Web service request can be for a single price, multiple prices, or simply for symbol validation against master data.

Hard to Maintain Market Data

If the market data in question is stored and refreshed often due to daily activity, such as historical time series and tick-by-tick data, then it can entail a complex update process that must be maintained and monitored. Quality testing must be put in place to ensure data quality and alerts to ensure that update processes run successfully to completion. As market data accumulates, regular backups, purges and capacity upgrades must be carried out to ensure efficient operation. Also, market data that is particularly complex, such as corporate actions, can consume significant resources scrubbing, mapping and updating the data even if the volume is not as heavy as price data.

On the other hand, market data that is infrequently updated in large batches that replace the entire data set may be easier to receive and maintain internally as a simple flat file. Similarly, market data that is streamed for continuous presentation and immediately discarded should be relatively easy to handle in-house.

Hard to Access Market Data

Technical and geographic barriers can conspire to make certain kinds of market data extremely difficult to access, let alone store and use. Getting access to market data that is not in high demand in your geographic location can be very difficult when local data providers do not support it, such as low volume market niches and new products, or market data that is created on the other side of the world. If data needs to be made available throughout a dispersed global organization, it can be quite costly to receive it centrally and build the necessary network and services infrastructure to distribute it globally to consuming applications. Cloud computing provides direct application access to market data over the Internet, so both the geographic and technical barriers to access are significantly reduced. Web services standards ensure that the technical hurdle is very low, so as long as sufficient Internet bandwidth is available, access is no more difficult than pointing a Web browser to a Web page.

The Cloud Computing Sour Spot for Market Data

If hard to access, hard to maintain, and hard to use define the market data cloud computing sweet spot, then what is the sour spot? When boiled down to its essence, all of the “hard to’s” above are made “easy to’s” by the Internet. So, the sour spot is found when the Internet has a significant negative impact on market data delivery. The three biggest limitations, in most likely order of importance are…

Too Risky

The market data is unique and proprietary with significant competitive, security or privacy concerns that preclude storing or distributing it using a public network, even with encryption.

Too Fast

The latency requirements of the data are very strict and have a low tolerance for variation. Typically less than about 50ms on average which is about the best that can be consistently expected from a high performance Internet connection. In addition, Internet latency can vary significantly depending on the number of network hops between the cloud provider and the consuming application. This number improves every year (back in 2000 we were talking seconds). But it is safe to say that latency requirements measured in microseconds or nanoseconds won’t be a good fit for the cloud.

Too Fat

The market data volume involved implies a network transfer time that exceeds the requirements of the consuming application given the best expected performance over the Internet. This may be alleviated by getting a direct network link to the cloud provider if the cost of the additional bandwidth is justified by the overall cost savings from outsourcing the relevant data management infrastructure. In some cases, this can also be resolved by moving the application itself to the same cloud computing infrastructure. However, an in-house application that requires frequent updates with large volumes of data can quickly exceed the best performance currently available over the Internet.