Financial Market Data Web Services – Xignite Blog

A few weeks ago, I was part to a discussion with Frank Kenney from the Gartner Group. He was talking about the need of providing greater governance transparency with web services. He recollected the enormous backlash SalesForce.com experienced last year when their system went down for a long period of time. As a remedy, and as as way to regain lost client confidence, SalesForce created a new web site (trust.salesforce.com) focused on providing its clients with complete transparency on the state of its systems.

Frank commented that this approach not only enabled SalesForce to yet again set the standard on how Saas should be done, but it built significant goodwill with its clients. SalesForce clearly showed it had nothing to hide.

This issue is equally significant for Xignite as more and more clients run more and more mission-critical business process on our services. We have historically achieved great levels of availability. We have nothing to hide. This is why we are busy at work providing a similar functionality to our enterprise clients.

Meanwhile, I was wondering if we could not do something simple that would help our clients easily answer the “are you having problems right now?” question which we get once in a while when a client has a problem accessing our services. 99% of the time the problem has to do with configuration changes on their side but I don’t blame fir first wondering if the problem is with us.

Here comes Twitter. Can Twitter do to web service governance what it has done for individual: bring total transparency on the availability of our systems?

You be the judge. Our support pages now boast a twitter-powered System Status widget that provide 5 minutes reports on system availability. If our primary web service farms runs into problem then the status is instantly be updated.

The great thing about twitter is of course that it lets any of our clients subscribe to our twitter in case they want to be updated in real-time of any changes to our system. Now that’s transparency.

I will be attending the Web 2.0 / Enterprise 2.0 in the Capital Markets Industry conference in NYC on 9/17 where I will moderating a panel on Enterprise Web 2.0 in Financial Services. This is a small event but it attests to the growing interest the financial services industry is giving to the new wave of technologies that have hit the enterprise over the last 2 years.

One of the most interesting ideas that I have seen emerging around this subject is a concept I call Private Service Oriented Networks. Here is how it goes:

Financial service providers (FSPs) have long provided their institutional clients (money managers, pension plans, etc.) with technology layered on top of their financial services. For example, with custody services, they provide analytics on top of the clients portfolio holdings. For execution services, FSPs provide services and analytics covering the pre-trade and post-trade processes.

The point of providing this technology is to increase the firm’s differentiation and the "stickiness" of the services–which are often very commoditized. In the "old world", such technology took some pretty basic form: daily data feeds made available to clients–mostly through FTP, at best some web applications. This method of delivery made sense because the clients themselves were running standalone client/server applications. Clients needed files to upload data to their own systems.

Although primitive, those technologies delivered quite a bit of stickiness. During my days at Advent Software, I remember talking to some clients ready to move several hundred million dollars to a new custodian solely based on the availability of an interface between that custodian and the firm’s portfolio accounting system.

But under the new Service Oriented model, this old way of doing business falls apart and many FSPs know it. I have been talking to more and more firms who are trying to find ways to "upgrade" the way they deliver technology to their clients for the brave new service-oriented world.

What they want is provide their clients with a collection of unique, private web services allowing them to build more powerful, more responsive applications. Forget about the old daily downloads, we are talking intra-day information here, whether it’s real-time portfolio or trading analytics, the goal is to enable the institutional client to easily access all the data and functionality they to build their new generation applications.

So imagine such a Private Service Oriented Network providing you–the institutional client IT executive- with market data, account information, execution services, accounting services, analytics services. All through an open web service (SOAP/WSDL or REST/WADL) interface. Suddenly, your ability to deploy service-oriented business applications greatly increases. And to you, the benefits are significant: rather than building everything in house, you can start treating your FSP as an extension of your own IT organization while preserving complete flexibility and control about what makes your business unique.

This is a bit similar to what SalesForce.com has done by exposing its internals through web services. You could say that the SF.com API was the first and largest of the Private Service Oriented Networks. Now thousands of clients have built deep integration between their internal systems and SalesForce.com. This makes switching CRM system hard.

I foresee that over the next several years FSPs large and small will be deploying their own web service networks. And just like in the old days having or not having an interface to your clients system could make or break a deal, tomorrow’s institutional assets are likely to flow in the direction of the FSPs with the best networks in place.

I am a late comer to the SOAP vs. REST debate, mainly because this blog started after the battle began raging. But a recent post by Raymond Yee–who is writing a book about mashups–suggested a reaction.

Raymond’s main point is that SOAP requires a developer to have deep knowledge of the protocols (WSDL and SOAP) to use web services effectively and that it is "so much  easier to get started with REST."

I disagree violently. The knowledge of SOAP and WSDL you need all depends on the SOAP toolkit you use.

I’ll take an analogy; Do we require internet users to know HTML to navigate the web? No we don’t. On the client side, the browser takes care of turning HTML into beautiful usable web pages. If end users had to read HTML pages in plain text they would find the web very obscure indeed.

On the server side, if you are writing your HTML by hand, you will definitely need to know the details of the protocol. But if you use a good advanced IDE that lets you write pages in a WYSIWYG fashion, you can build a web site without knowing anything about HTML.

So it all depends on the tools (client and server side) and how well they hide the protocol from you.

If you want to write a web service by hand, you will have to know SOAP and WSDL. If you use a good toolkit, all you have to do is write a function call and you have a web service. If you want to consume a web service without a good SOAP toolkit, you will have to learn the standards but not otherwise.

We have written dozens of web services which collectively have served billions of SOAP requests to hundreds of firms around the world yet we have never read the SOAP specs and we can barely read a WSDL by hand. All we had to do was to write meaningful function calls (which is an art in itself).

Our clients who use good SOAP toolkit can go from pointing to our WSDLs to consuming data in their application in less than 60 seconds without a hint of what the protocols are about. This is not something you can do with REST yet (WADL might be able to change that).

Clients who come to us with more primitive languages will tend to spend more time working out the details of the integration.

It seems that SOAP’s bad rap comes from those who just have not wanted to use more advanced SOAP toolkits.

It’s very sad indeed. It is as if back in 1996 we had abandoned the promise of the web and stopped building web pages because a bunch of people insisted in using a text editor instead of a browser to open HTML pages and complained about how unreadable this HTML was. Come on.

To his credit, Raymond is very open to being convinced. He wants to "be shown where my thinking is wrong  and how I can start using SOAP and WSDL with ease in PHP, Python, etc."

Unfortunately, and with due respect to the fantastic tools PHP and Pythons are, they are not on the top of my list of the "better SOAP toolkits"–at least in their editor form. We had to handhold too many developers using those to have them make the cut. I suppose that if you use a nice IDE on top of those, things are easier. But they don’t require you to know SOAP and WSDL to make them work.

In a recent post, Dan Farber and Larry Dignan
talked about the fact that there is still much ground to cover before
all major Internet properties will provide the "five nines" (99.9999%
availability). Their key point was that today’s Internet services are not "ready" for the massive wave of utility computing ahead and that more money and intelligence needed to be thrown at those problems.

Their post talks about "web services" in more general
consumer sense than ours (where we mean XML web services), but the comment certainly applies. With more than 300 clients in 25 countries and upwards of 500 million
web service requests a months, our web services are powering more and
more mission-critical business processes buried deep inside the core
fabric of our clients’ business. This of course raises the question of the availability of those services and what’s right given the maturity of the market.

Web services as a "computing utility platform" for business are at least 10 years behind the web as a utility platform for consumers. That wave started in 2005 while the web goes back to 1995. So it’s normal to expect a lower level of maturity. Our experience to date has been that clients and prospective users have been more and more sensitive to availability issues but that guaranteeing "three nine" (99.9% availability) meets all current business needs. I expect that the market will be requesting another "nine" every year for the next 4 years (2008->99.99%, 2009->99.999%, 2010-> 99.9999% and maybe more).

There is one key reason for that. Few of our client business processes run themselves in a "fine nine" mode. Many processes are mission critical but they are mission-critical once or several times a day, giving plenty of time for retries and recovery. For instance, one of the world’s largest corporate lenders gets some interest rates information from us. This info is used to feed some homegrown systems that decide how much interest corporate clients gets charge for tens of billions of dollars in corporate loans. That’s mission critical. But the process runs once a day (replacing an old unreliable manual process). As long as our system is up within a 1/2 hour timeframe every day, all is well. In truth, we achieve more than 99.99% availability, with many monthly periods at 100% (we have been publishing availability data on our site since 2004. You can find it here). But it’s almost irrelevant to this client.

The key is to map the service level to the needs. Why going to the trouble of providing
99.999% availability if your clients don’t care at all or don’t care yet? Running redundant data centers is extremely expensive and difficult to do for an emerging company. Your clients probably want you to spend your money doing other things firsts. But once they begin asking for more "nines", you’d better get moving.

And so we are planning for our redundant data center–which we will need to get to the "five nines".

I recently had the opportunity to meet with the senior IT management team of one of one of the countries’ largest commercial banks. This was an open, round table discussion involving multiple Silicon Valley startup companies who—like us—try to woo similar-sized enterprises into using their latest and coolest wares. The CTO started saying that in the old days, standard software was supposed to meet 80% of a firm’s need, with the remaining 20% requiring customizations to fit the firm’s process. “With packaged software these days”, he said “it is more like 50%/50%. This forces us to spend a lot of time customizing things, while we are actually trying to be more agile and more responsive to changing market conditions.”

This comment struck a chord with me. Many IT executives now understand that re-usable web services finally offer the promise to address those changing needs. Instead of buying a standard app that only does 50% of the job, the firm can build a custom application from re-usable web services. If you can buy most of the services you need, you get the best of both worlds: the fit of
a custom app for less than the price of a standard app.

That’s all nice and good, but there is a big assumption built in: you must be able to purchase most of the services you need without having to customize them much. Otherwise you are back to where you started. And there lies the problem with the state of the web service industry today. Dana Gardner made it clear on his ZDNet blog last month: “There are either not many services available or the services are too general and not specific to a specialized vertical industry or a niche function.”. Dana further states that 80% of those services need to be customized. This type of problem could seriously hamper the success of composite applications in IT America.

Since we have been delivering web services for a while, we saw that problem coming a while back. This is why we not only offer a broad and deep set of integrated web services sharply focused on a specialized industry and function (financial market and industry data–in case you did not notice), but we also engineer those services with the needs of the composite application designer and developer in mind. Our goal is to make those re-usable components so integration-ready that you just have to drop them “as-is” in your composite applications. We still have ways to go, but we will get
there.

And what’s the payoff you say? Gardner again puts it best: “The higher [developers] can drive the percentage of reuse, the more scale and productivity they gain. They become the go-to organization for cost-efficient applications in a specific industry, or for specialized business processes.”