Friday, June 04, 2010

Someone Will Always Have the Data First

I'm going to keep this short and sweet.  Scott Patterson writes in the Wall Street Journal:

"Some fast-moving computer-driven investment firms are getting an edge by trading on market data before it gets to other investors, according to market players and researchers who have studied the trading.

The firms gain that advantage by buying data from stock exchanges and feeding it into supercomputers that calculate stock prices a fraction of a second before most other investors see the numbers."

Listen up now - someone will always have the data first, and someone will always have the data before you.  It's a fact of, well, data transmission.  The guy in San Fran will get the data a little later than the guy in Chicago.  The guy in midtown Manhattan will get the data a little later than the guy in downtown Manhattan.  The guy on a DSL connection will get the data a little later than the guy on the fiber connection.  The guy next door to the stock exchange will get the data a little later than the guy who is co-located at the data center.  The guy using the internet will get the data a lot later than all of them.

Again, the important fact is that this data is open to anyone who is willing to make the investment in it  - not just a secret cool kids club that requires you to work for a specific blue blooded Wall Street firm.  Anyone can do it - if they are willing to invest in the business.  

The "Hey - it's not fair - he's getting the data first and reacting to it faster than I can" argument is total baloney.  

EDIT - as I mentioned in the comments: also notice that 15 years ago the sophisticated market participants had real time pricing information, while retail schmucks were left calling an automated phone system and punching in their tickers to get 20 minute delayed quotes (come on - I know I'm not the only one who remembers that).  Today, less sophisticated investors have easy ample access to "real time" quotes which are in fact seen 100 MILLISECONDS after they are published, and thus 100 ms after the fastest market participants. So the retail data availability time lag has been decreased from 20 minutes to a number of milliseconds.  Sounds like improvement to me. 



Anonymous said...

Thank you! we pay to get data first. since when is that a crime?

Anonymous said...

Insightful thoughts. Why do you assume that getting data first is always an advantage? The reaction to news is much more important that the news itself. Different people take different time to analyze new data and they might come up with completely different conclusions. The guys who act after the initial reaction to new information, often have the edge of already established trend.
You can sit and complain that the game is rigged or you could figure out a way how to turn that into your advantage.

asphaltjesus said...


What you wrote is the equivalent of a grammer schooler's, "nuh uh!"

You purposely avoided addressing the negative consequences of tiered access as illustrated in the article.

How about explaining the benefits the customer gets in the scenario given? Or, maybe there's a problem with the scenario as described in the article?

Anonymous said...

Come on. This argument is beneath you. It isn't that someone gets the data first, it's that people are allowed to step in between orders as well as get a look at others' order flow. That is frontrunning. Period.

Kid Dynamite said...

Ivanhoff - i'm not sure if you were addressing me, but if you think that i'm complaining that the game is rigged, you need to read more of my blog. not even close.

asphaltjesus - tiered access? LIFE is tiered access - that's the ENTIRE point. YOU can get whatever access you want. It requires capital, skills, and investment. That's called capitalism. It's like complaining that I can't compete with Kobe Bryant because he's better than me. HIT THE GYM! It's like complaining that i can't enter my 2000 Honda Civic in drag races because it's too slow - GET A NEW FRIGGIN CAR. not everyone and everything are equal.

BUT - the important thing is that everyone have EQUAL OPPORTUNITY - which is exactly what I explained in simple words. YOu, AsphalyJesus, can get a direct data feed. you don't have to work for GS, or some massive HFT firm. You yourself can get it. Oh - you can't afford it? TOUGH LUCK - that's how business works! It takes capital investment.

If you think that you, sitting at home, have the right to whatever data anyone else has for free, well, i suggest you find another site to troll that opinion on - nothing productive can come from arguing that with me.

now, what are the negative consequences? scott patterson (WSJ author) is clearly not a trader. he gives an insane example about how TFS was willing to pay $70.49 for a stock, and guess what - THEIR BID GOT HIT WHEN A SELL ORDER CAME IN! it happened to have been their own sell order too, which is ludicrous on TFS's part, as it could be prosecuted under painting the tape. which brings me to anon's comment - again - the trading example is ludicrous, and it has absolutely NOTHING to do with frontrunning. It has to do with the fact that the consolidated tape is slower than the direct feed in reporting trades WHICH HAVE ALREADY HAPPENED!

patterson then spews nonsense about how they paid too much for the stock, which is utter crap. they had a bid. their bid got hit when it was the best bid in the marketplace. that's how the market works.

Dark Lord Mandelson said...

"Frontrunning" allegations and all that are beside the point. The important question is: Is a market in which multimillion-dollar investments into fibre and stock exchange colocation determine financial success for a liquidity provider increasing public welfare? If not, the rules should be changed.

Obviously this is an ever-escalating zero-sum arms race of computing power between the HF trading firms, and of zero value to actual long term investors (who really really don't care whether their order gets executed 10 or even 1000 ms later at the same price). What would keep the big exchanges of going for bid-matching every second, or even every 30 seconds? It would certainly decrease investment into totally useless (though lucrative) endeavors like this!

Kid Dynamite said...

ps - Anon - no one is getting a look at anyone else's order flow, despite what you've been mislead to believe. THAT would be frontrunning.

pps - Asphalt Jesus - sorry for the fired up reply - you got me all fired up. Yes - to the last part of your question - the example cited is massively confused. TFS's bid (LIMIT ORDER) got hit when it was the best bid. that's how it works.

also, again, TFS's activity is walking the line of legality - you are not supposed to enter an order which can be executed against another order you have outstanding (like the sell order they sent which crossed their bid). anyone have the relevant law code on this? i'm looking for it...

Kid Dynamite said...

dark lord: a few points

1) you don't notice a benefit in public wellfare, but how about all the firms selling the technology equipment? they might disagree with you - they certainly benefit, even if it has little to no effect either way for investors in the market. (although i'd argue there is a positive effect via decreased spreads, but that argument may not apply to latency arb in particular).

2) so let's say we ban colocation. The entire point i was making is that someone will always get the data before you, even if you ban colocation.

Anonymous said...

You're tearing down a straw man (albeit a straw man in which a lot of people errantly believe. The problem is that, de facto, a large percentage of our population has been forced to participate directly in asset markets as businesses have migrated from defined benefit to defined contribution plans. It's the only way they can ever hope to retire given the massive inflation in health and education costs driven by the other two major government supported cartels (outside of Wall Street). And a large percentage of those people cannot meet the necessary barriers--even if they work with firms who do engage in HFT--to get data rapidly, on even footing. So, yes, it is true that the option is technically available to anyone who wants to participate. However, the people least likely to have the resources-- whether financial or knowledge-based--to get data with comparable speed are the people most likely to be forced, against their will or better judgement, into competition with those who can afford the data. And then well meaning, smart guys like you explain that, hey, if they only devoted their lives to the financial arts, they might be able to compete... well, it's unreasonable (and, frankly, a bad allocation of human capital) to suggest that everyone must dedicate his or her life to financial machination... in part, it's what landed us in our recent predicament.

Anonymous said...

sorry for the continuation from my previous defense of the "regular" guy-- but it is also disingenuous to act as if the people who can afford great data access have not benefited tremendously from the regulatory and structural push to drive the "regular" guy into the market--even if we discount the winning trades. Financial service firms sell $2T worth of products and services to US consumers every year-- and then they use the $2T to buy co-located servers and say, "sorry you lost your money... maybe if you had more, you would have won!"

Kid Dynamite said...

anon - YOU just erected a straw man! NO ONE is forced to trade anything. and guess what - this really only matters to TRADERS. if you're a long term investor, making allocations to your 401k, this has no effect on you. really. i know you think the computers are front running your trades, but they're not.

markets have evolved. When you place a bid for stock, other market participants, humans, computers, or otherwise, may see that bid, react, and place a higher bid. there is absolutely NOTHING wrong with that.

Kid Dynamite said...

Philip Hwang has some intelligent comments on the comment thread of the WSJ piece. here's one

"It's front running if you intercept an order on its way to the exchange because that information is still private since only the person submitting that order knows about it. But if the order reaches the exchange "unintercepted", then the exchange disseminates this information, it's public information at that point. And at that point, the fastest wins.

Back in the days, the specialists saw it first, then the floor brokers, then ultimately the retail investor after calling his retail broker. Now the guy connected via a 10-ft fiber optic cable sees it first, and ultimate the retail guy on a dial-up. Whatever the event, there's always a sequence of dissemination; a retail investor needs to either upgrade his technology or change his way of trading. But I suppose we can hold out for hope that Congress bans highspeed internet and caps bandwidth at the equivalent of a 28.8k modem for everyone. Even at that point, what if the investor wasn't paying attention because he was concentratin on another trade? Should the world wait for him?"

Kid Dynamite said...

anon@ 2:29: sorry, i'm all over the place - this issue is so simple, and yet so many people have had their brains turned to mush by propaganda and have no clue what the facts are... you wrote:

"And then well meaning, smart guys like you explain that, hey, if they only devoted their lives to the financial arts, they might be able to compete..."

on the contrary. I don't suggest anyone devote their lives to this - there will always be someone better than you. again - if you're an INVESTOR, this has little to no effect on you, or even helps you (by the way, in case you missed it:

if you want to be a trader, then guess what - you're trading with the big boys! you have ZERO right to success. You have the right to the same opportunities everyone else has - which is exactly the situation. You want to co-locate? go for it. BUt don't sit at home with your Schwab account trying to day trade and complain that others are faster than you.

Kid Dynamite said...

anon @ 2:33 - sure - the system sucks, etc. (the whole Stock Market Machine - the system to get people to invest - sucks). i can agree with that to some extent. technically sophisticated markets which have progressed massively over the last 20 years are the LEAST of the problems in the system -despite the subterfuge being propagated by the haters.

Anonymous said...

KD- it's your blog, so you can definitely win if we want to throw straw man accusations back and forth. I won't keep sniping at you after this comment (because, in general, I love your work).

That said, people ARE de facto forced to participate. The average American makes so little money relative to the cost of living that, in order to support his or her lifestyle in retirement, he or she must invest in the market. And this is not a crazy or new phenomenon. Previously, large pension funds with better collective resources handled that effort for the average guy. Now, that is not the case. And, in fact, Wall Street has profited greatly from the disaggregation of consumer retirement assets. SO, since these people are forced to put money at risk (at minimum, to defend against the destructive effects of printing money to re-paper banks' bad bets--why else do those of us in the business consider low interest rates and QE "forcing people out the risk spectrum"), it is unfair to tell them that they cannot compete on even footing unless they are richer. In addition, the idea that long-term investing will always work is born of an era of increasing systemic leverage-- ie 1982 - 1999-- for the last decade, it has failed investors severely, and, with the proliferation of both electronic and human arbitrageurs alongside massive government intervention in the financial system and the high likelihood of private (if not public deleveraging), the argument that average Joe should just buy SPY is roughly the same as saying he should just spring for a co-located server.

So, yes, you have a straw man on your hands... it just happens to be an argument people normally do not consider as such because they do not think about the motivations of most people for participating in the market. They have to-- as they always have. We've just taken away their safety in numbers.

Anonymous said...

Unfair! I am not saying "the system sucks, man-- get me off the grid-- move me up to New Hampshire to make syrup!!" Shit- I run a lot of money! I am just saying it IS a rigged game-- as it has always been-- but by making the market more rather than less fair, we all benefit from greater efficiency. If we are going to point to academic papers- read some Daron Acemoglu- once people decide the system is rigged, the cost of capital for everyone goes up, and efficiency drops considerably. Perhaps we're seeing this now? Maybe not... but eventually... The desire to make things BETTER doesn't make me anyone a hater-- I think that is a really unfair comment--

Anonymous said...

With Hungary now the economic pain point du jour, we should add Hungary and change Italy from I to It. That makes PIIGS change into PIGSHIt. That's more like it.

Kid Dynamite said...

anon - you are losing me. let's pretend that people need to be "in the market."

are you now trying to argue that buy and hold is dead, so the average investor must have the right to actively TRADE his 401k on equal footing with high frequency traders? that we should slow the market down so retail guys are on equal speed? i don't know many educated market participants who would agree with that. i'm also guessing this is not what you're saying, so sorry for putting words in your mouth, but you lost me.

if you want to rebalance your 401k 4 times a year, or even 6 times a year, latency arbitrage will NOT be eating your lunch. Seriously - true.

put in your limit order, and wait for it to get filled. you can even enter a bid above the last price as a LIMIT. nothing wrong with that. you set your own prices - no computer can force you to pay more or to change them. other people CAN be wiling to pay more, though - and there's nothing wrong with that

Anonymous said...

Are you serious? Put in your limit order? How about your stops this May? Buy and hold is not dead; it is much more difficult than simply putting together a diversified allocation of index funds and rebalancing periodically. that is why, according to most observers, a great proportion of Baby Boomers, who worshipped at the buy-hold-rebal alter for the last 20 years, are financially unprepared for even modest retirement.

And, yes, the reason the majority of people (not the majority of capital, but the majority of people) invest in the market is because they must. Without that appreciation, they'll be greeting you at Wal Mart until the day they die.

Kid Dynamite said...

too many anons - guys - sign your posts with initials or codenames or something so i know who i'm talking to. i can't tell if i'm talking to 1 person or 5.

anon @2:51- 1) i wansn't calling you a hater - but there are a lot of vocal, conflicted, HFT haters out there who are brainwashing the public with misinformation.

you run money? guess what - latency arb doesn't effect you - unless you run a highly active trading operation - and if you do, guess what - there are others who do it better! that's all i meant.

if you're a money manager, and you decide GE is worth $15, and put a $15 bid in for stock - all the computers in the world running all the super latency arb or other strategies in the world aren't going to effect you.

Anonymous said...

As KD says, the data is there for anyone who wants it, and it is fairly cheap.

As someone who runs a 1-person HFT shop, I can attest that under $10K in hardware can process the entire raw data feed from the major exchanges and ECNs in the US (if programmed efficiently in a low level language like C), and the costs of the co-location space and the raw feeds will add several thousand dollars per month at the outside (and can be much cheaper if you are negotiating with a co-located broker-dealer to use their space and existing data connections).

It is fun to read rants that this type of connectivity and processing power is only available to large teams of programmers with millions of dollars in hardware, but it is simply not the case.

There is an intellectual hurdle - namely being able to program well or hire someone who can do the job for you, but that is just capitalism at work. Those with the right skill sets to profit from an opportunity will do so.

Market making has never been about understanding fundamentals of the securities being traded, but rather to act as a lubricant for the market matching natural buyers and sellers over different time periods.

Those who argue that the markets would perform better without automated market makers ignore the long history of gouging by human market makers trading at much larger spreads measured in fractions...


Kid Dynamite said...

anon @ 2:56 - i've written at length about stop orders and the may 6th crash. you can go read all my posts on the subject - you'll hate them. but ignorance of market mechanics is no excuse. i've written at length about easy ways to protect the ignorati from themselves.

you are getting into a totally different topic. i said i'd acknowledge that people" have to invest" - even though it's not totally true. let's take it as an assumption. people do NOT have to try to compete in a latency arb trading strategy.

look - we have NBBO protection rules - they are the most important rules in our market. that's what really matters - no one is selling stock at $14.95 when you have a $15 bid out there. when your bid is the best marketable one, it gets hit. When your offer is the best marketable one, it gets lifted.

Anonymous said...

PeterPeter & KD- I am not arguing about HFT--I have no problem with HFT-- I am arguing against the spirit of your comment--"The 'Hey - it's not fair - he's getting the data first and reacting to it faster than I can' argument is total baloney." It's about data availability- not data use. And it's not baloney.

You argued that premium data access is a fair aspect of a free market. I pointed out (in many more words, apologies), that the community as a whole benefits from more efficient markets, and efficient markets have low/no barriers to entry or exit. Given that the community is required to backstop the markets lately, I think it's general benefit should be considered.

In the case of the stock market-- most people CANT get out--and, of course, the data barriers are what they are. Doesn't really, in my mind, have anything to do with, the latency arb argument is yet another straw man... It's about efficiency-- and trapping people in a game in which they have fewer tools reduces efficiency. I don't care what PeterPeter's great intellect does or does not allow him to do with his boxes.

AA - initials, per your request

Anonymous said...


The markets of course would be more efficient if co-location space were free, everyone were given a free computer capable of processing the raw feeds, and everyone had the exact same super fast feed handlers connected with the shortest fiber links direct to the matching engines.

However, for anyone who cares about such things (which is certainly not investors!), it is there for a reasonable price. As businesses go, the barrier to entry in market making and stat-arb are incredibly low.


Kid Dynamite said...

more importantly, AA - you wanted to talk about data availability. 15 years ago, you got 20 minute delayed quotes, while all the sophisticated market player got real time feeds. You were 20 minutes behind.

now you get real time quotes, which are actually, essentially, delayed by 100 MILLISECONDS - because someone else is seeing them 100ms before you.

sounds like a pretty awesome improvement to me.

Anonymous said...

PeterPeter- you are totally correct and, in general, I agree with you. But that's not my point. My point is more about the barriers to exit (although the average guy with a 9-5 and an IRA a. does not have the training, time, or money to surmount even those barriers, and b. probably cannot convince his financial institution to allow him to place money with your firm).

I don't really care how the data is USED. I care how it is distributed. If people are forced to play with a short stack, efficiency is dropped, and, in the long run, the cost of capital rises for everyone. All the inflationistas in blogland are constantly wringing their hands about rising costs of capital--well, this sort of inefficiency is one of the core sources thereof.

Just because data tiering is possible and, in some applications such as HFT, provides some ostensible benefits, like volume (I would, of course, argue that volume and liquidity are different things, which is why emerging markets prefer FDI to market investment) doesn't mean it is broadly beneficial...or even that it is not detrimental to the general economic state


Anonymous said...

Kobe comparison? Really?

Drag races too?

You make it sound like in capitalism country, if you can afford it, cheating is acceptable.

Kid Dynamite said...

cheating? really Anon? come on - that's just a troll comment and you know it. there's no cheating going on here, as i've already typed thousands of words explaining to you.

equal opportunity is not the same as equal success. sorry you don't like that. my kobe comparison was probably stupid. the drag racing one is quite relevant.

in capitalism country, EXCELLENCE is acceptable. SUCCESS is acceptable. "BETTER" is acceptable. it's encouraged, in fact.

Anonymous said...

KD- on an absolute basis, I agree, the situation has improved. But 20 years ago a much larger percentage of the population participated in markets through massive pension pools and, thus, enjoyed better relative access to information pertaining to the majority of their risk assets. And people (like dogs: judge their well being on a relative basis. So while the improvements are good, the problems represented by these availability gaps have not dissipated (not sure I have enough data to argue better/worse, though my gut says worse- don't kill me for that, though, I admitted I don't have the stats to back that up!).

Moreover, as Bernanke indicates in some of his most basic macro writing and in his behavior, much of macro is perception-driven. And the perceptions created by this gap in availability are bad. So that should increase cost of capital in the long run. And increased cost of capital due to inequality is bad for everyone!!


scharfy said...

Here's a thought experiment for all the doubter's out there:

What to you think would happen to sports betting lines in vegas if we legalized them and let Wall Street and main street compete on NBBO for the action?

Yup we would get better lines.

I'd be betting the Bears +7.27467 with juice of 6$ per trade regardless of size.

Let the markets evolve and let the computers chase the nickels and pennies, its better for us.

scharfy said...

bad example on the +7.25757 . Thst would still be the same as 7.5 - but ya get the point~!!!

Reformed Broker said...

I'll weigh in and remind the crowd here that this is the same shit that Nathan Rothschild used to pull, he had a 48 hour jump on the information from the Waterloo Battlefied and converted all his hoarded gold into british bonds before Wellington's account was read in parliament.

His couriers were better fed, positioned and trained than anyone else's courriers and the Rothschild's owned the world because of it.

Sound familiar?

Kid Dynamite said...

lol Scharfy with +7.25642 points ;-)

AA - you've touched on a very important point which is actually a main part of what i've been trying to SOLVE... PERCEPTION. that's exactly my point - the REASON that rubes and noobs are losing faith in the markets is because they read all this scary stuff about dark pools, high frequency trading, milliseconds, yadda yadda yadda - and you know what - none of it harms them.

that is my point - the perception is driven by hypsters, others who don't understand, mainstream media authors who want page views, bloggers who want page views, others with agendas (like brokers and antiquated day traders who can't compete with HFT) - but the fear is totally unwarranted. there has never been a better time to be a small investor in terms of ease of access, availability of information, cheapness of execution, and tightness of spreads - i already wrote a whole post about that.

if you think that you, as a retail investor (i'm assuming, since that's who we're talking about anyway) are worse off now than you were 20 years ago, you're flat out crazy. we've replaces specialists skimming quarters with machines skimming FRACTIONS of pennies.

(if you haven't read this post, please do so:

anyway, perhaps when i said that the "that guy gets the data first" argument is baloney, what i should have said is that it's a TRUISM. it's always true - no matter what - that someone else gets the data before you - as I illustrated in the post.

EconomicDisconnect said...

Out of hand much?

Anonymous said...

I think KD is right. When market information was conveyed verbally, some listened and acted faster. When it switched to pencil and paper, some could read and write faster.

There will never be a market where the quick are not the quick. The light turns green (information becomes public), somebody is quicker and maybe wins. There is no way to make it equal. There never has been.

Anonymous said...

There are two claims that I find at odds:

1) High Frequency Traders do no injure the small investor

2) High Frequency Traders had a net profit of $20 billion in 2008.

Over the very short term measured in seconds, equities are a zero sum game. So if one HFT firm is making money, someone else is presumably losing money. If it is NOT another HFT firm on the opposite side of the trade, who is left other than the investor NOT using HFT?

If the HFT traders have a net profit of $20 billion, it would seem that money has to be coming at the expense of the NON HFT trader aka the small investor.

Anonymous said...

Anonymouns @3:19 AM

> If the HFT traders have a net
> profit of $20 billion, it would
> seem that money has to be coming
> at the expense of the NON HFT
> trader aka the small investor.

While it is difficult to differentiate between the profit contribution to HFT from retail vs. institutional traders trading on behalf of retail vs institutional traders trading for non-retail, your point is valid. The profits are coming from somewhere...

Much of the profit comes from the exchanges, which offer rebates in the maker-taker model, but then even with the higher volumes attributable to HFT, it is clear that the maker rebate is getting passed on to other participants in a higher take fee (which in turn pales in comparison to the broker fees most traders pay!).

However, 3 points:

1) Retail investors who trade infrequently (and investors as opposed to speculators do trade infrequently) after doing their research on positions and setting limit orders for the price that they wish to pay for a particular security I would argue are incurring no fee from HFT, and to the contrary are seeing tighter bid/ask spreads and lower commissions due to the increased volumes from HFT. So, a retail investor trading on their own account I think certainly benefits from HFT. There is no way to anticipate a retail order, since it is "one and done", and no computer system can become smart enough to know when the whim of a retail investor will make them place their limit order. Since there is no follow on order, they have nothing to lose to HFT, but much to gain.

2) Retail investors who have mutual funds that are trading with technologically dated tools which makes order anticipation by HFT possible are being harmed somewhat by HFT. However, 2 sub-points

2a) These same funds passed on much larger trading costs to their investors in years past with human market makers trading in fractions and colluding. See the following for a brief history lesson on recent past:

Markets need market makers to function, and the harm caused by these HFTs due to increased costs is only against a theoretical comparitive world in which no market makers exist but trading somehow still works! In a world of market makers, there has never been a lower cost incurred than via the intensely competitive HFT world.

2b) The increased costs of investing through a mutual fund from HFT (ignoring that it is still cheaper than days past with human market makers) pales in comparison to the management and marketing (12-b1) fees that these investors pay. Mutual funds should be relegated to the waste bin of history along with human market makers, as they only enrich the fund managers rather than the investors...

3) Most people trading are speculators (whether directly or indirectly through funds). People who do not vote their shares, do not read the 10-Qs and 10-Ks, do not look at dividend ratios, Q ratios, P/E and P/B ratios are NOT investors. They are speculators, hoping that someone will just buy their shares from them at a higher price. Those folks will on a statistical basis and taken as a whole lose to HFT and stat-arb funds and sophisticated human investors, and they have no right to do otherwise. They have chosen to enter a casino rather than a market which allocates capital efficiently, and they are going to lose money.

Now, I do feel for the average lay-person who is socking away some money into a 401K that in turn is allocated to a bunch of mutual funds (and may not even realize that they aren't investing). Those people really should not have money in the stock market, but instead in savings accounts with positivie real yield and the bond market. That however is a topic well beyond the scope of KD's point about data access.


Steve said...

have you heard about "sub-pennying?"
In my opinion, it undermines the nbbo.
Here's an article that discusses it in depth.
I'd love to hear your opinion on this.


Kid Dynamite said...

anon @ 3:19 am - i'd add that numbers which seem like a lot, like "$20B" or whatever number we want to throw out for HFT profits are nothing in the grand scheme of things when you figure out how much of that you're actually paying - which as I've said many times - is fractions of a penny per share. Yes - the profits absolutely have to come from somewhere - they come from the pockets of all the old school market makers and traders who used to do what the HFT guys are doing - but they used to do it slower and at wider spreads.

Steve - sub pennying is annoying, but it does not violate or undermine the NBBO. the important thing, with the example in the first paragraph of the piece you linked, is that when you have a $25 offer, the stock doesn't trade ABOVE $25. In the example, the stock traded at $24.9999 (in reality, the actual fill would probably be $24.995) - so the $25 limit order from the seller was not violated, which is essential.

This would lead us to a discussion of a whole bigger issue - payment for order flow - which explains why you see such prints - I see them 85% of the time i trade in my account with Etrade - I get 1/2c price improvement. This is a function of fragmented markets, and as i've said many times, if we had one big dark pool to trade in, instead of a few handfulls of connected market centers, i think it would be an improvement.

Anonymous said...


I'll take the other side of the sub-pennying issue.

While it is certainly legal, I think the net effect of sub-pennying is a little bit counter-productive (and counter-intuitive) towards market efficiency and price discovery.

Of course the order being improved (even if by 0.0001 per share) is a benefit to the person making the order, but the improvement is quite nominal.

On the other hand, the advantage of broker-dealers handling the order to be able to sub-penny improve an offer means that other market participants have increased adverse selection in their trading.

Said a different way, the orders most likely to be "improved" are retail orders that are market orders or mis-priced limit orders. They represent the juiciest order flow against which to trade, as taking the other side is more likely to be a good bet than against say a well placed stat-arb fund's order. This is why firms will pay for the retail order flow from the likes of E*Trade and TD.

When building a model to make a market in a security, it is important to understand the effects of adverse selection (getting filled with orders that others ahead of you in the queue or broker-dealers in the middle didn't want, and therefore getting fewer of the more-likely-to-be-profitable orders sent your way).

Once adverse selection is taken into account, the net result is that a wider bid/ask spread needs to be produced by the market makers. That in turn thwarts price discovery somewhat, and increases somewhat trading costs for orders that are not price-improved by a BD in the middle.

It is a very hard analysis to perform (I think a long test on a selection of securities exempted from B/D price improving would need to be performed), but my hunch is that overall bid/ask spreads would narrow to the benefit of more retail traders if sub-pennying went away, more so than they currently benefit from the sub-penny improvements.


Anonymous said...

KD, I would also point out that the 20billion off HFT sounds like a pretty high figure to me in 2008. Remember this guy is claiming that as profit. I wonder how HFT is defined to get that figure and I'll put a certain amount of money out there to say he mean revenue.


Kid Dynamite said...

Danny - of course it SOUNDS like a big number - that's the scary part... break it down per trade, though, and it's irrelevant to you as a trader

Peter - I'm not about to start writing blog posts defending sub penny pricing - i'm somewhat indifferent on the issue - i was merely pointing out that the NBBO was not violated.

I definitely think that penny pricing is better than teeny pricing (1/16th) - so it's kinda logical to conclude that subpenny would be even better, but i understand that the benefit gets outweighed by other factors at some point.

I was actually thinking that if we allowed sub penny pricing but NOT payment for order flow, then spreads would probably narrow further - as those "improvements" in price would have to be incorporated into the NBBO. This might require consolidation into a single market center too though.

Anonymous said...

Here is the reference that says HFT's earned $20 billion in NET profits in 2008. The $20 billion is not gross revenues.

"In 2008 market centers paid over $2 billion in liquidity rebates alone, with most of these payments going to HFT market making firms, while HFT firms as a group were estimated to take home $20 billion in profits. As a comparison, in 2000 NYSE specialist firms had slightly more than $2 billion in total gross revenues."


This is a document that was written to in response to a SEC "Concept Release on Equity Market Structure." Here is the link to the original SEC post:

I agree that $20billion when taken on a per share basis is NOT that large a number, however it is disingenuous to say that HFTs do not harm the small investor. Whether you call him an investor or a speculator, he is paying this $20 billion.

Kid Dynamite said...

yes - again - of course the $20B comes out of "the market"... nice data point on the $2B from specialists. So who pays the rest of it? Sure - the small investor pays some (but less than they ever have before), and the small trader pays the rest - in other words, it's not like "trading profits" in 2000 were $2B - that's just specialists. Then there are hundreds of thousands of other "traders" in the market who have been displaced by HFT. I have no sympathy for them.

the focus here is on the individual investor, who isn't trading daily, and again, i'll argue at length that the effect of HFT on this individual retail investor is at worst negligible, and at best beneficial, for reasons which I have outlined many times (the biggest being tighter bid/ask spread).

Kid Dynamite said...

anon - to clarify re: $20B - i absolutely understand why these numbers concern you. What i'm saying is that they are kinda a bait and switch sort of fear inducing data point. in the year 2000, it's impossible to say how many traders were making how much profit - you found a good data point on $2B from the specialists alone. There may have been 100,000 day traders (or professional floor traders, etc) scalping $100k each in the market... that's what - $10B right there? who knows - i'm just making up numbers - you can make up your own. if you don't like mine. THOSE are the guys who are having their lunch eaten by HFT - and so what? it's technological advancement. No one has the right to trading profits.

The point is that the retail "investor" always pays a vig to the traders in the market, and that now the vig is indisputably (in my opinion) lower than ever. bid/ask spread is the vast majority of this anon - to clarify re: $20B - i absolutely understand why these numbers concern you. What i'm saying is that they are kinda a bait and switch sort of fear inducing data point. in the year 2000, it's impossible to say how many traders were making how much profit - you found a good data point on $2B from the specialists alone. There may have been 100,000 day traders (or professional floor traders, etc) scalping $100k each in the market... that's what - $100B? who knows - i'm just making up numbers - you can make up your own. if you don't like mine. THOSE are the guys who are having their lunch eaten by HFT - and so what? it's technological advancement. No one has the right to trading profits.

The point is that the retail "investor" always pays a vig to the traders in the market, and that now the vig is indisputably (in my opinion) lower than ever. bid/ask spread is the vast majority of this "vig" and it's impossible to argue that it's not significantly tighter than it ever was.

asphaltjesus said...

Anon @ 3:19, this is the heart of the HFT matter for me.The money in HFT is coming from some other market participant.

Most individual investors (buy and hold) feel like they've been robbed at this point. (especially after the roller coaster from a couple of weeks ago.)

I can corroborate the hosting and coding claims made by the individual HFT. Colocating is not 'expensive' relative to the overall costs of pursuing a HFT strategy.

I'd appreciate some more details about where the scenario in the article goes wrong. Ordinary people don't have any insight into the details of a trade, so the article seems plausible.

Kid Dynamite said...

asphalt jesus - i already explained what's wrong with the trading example in the article. The trader's bid was hit when it was the best bid. that's how markets work. there's nothing shady about it at all.

don't get sidetracked and confused by the end facts which are this: 1) as i explained in the body of the post, someone will ALWAYS get the data before you. it's a fact of electronics/physics/bits and bytes. 2) you have never been able to trade easier or cheaper as a retail investor - i will argue this point until my fingers curl up from Carpal Tunnel. It's what really matters in the end, despite red herring data points about how much other market participants (like HFT) are making.

Anonymous said...

Anon 3:19 here again: KD said: "The trader's bid was hit when it was the best bid. that's how markets work."

KD, I agree that is the way it is supposed to work and usually does work. However, it does NOT work like that all of the time. It is not a HFT issue so much as a dark pool issue.

If you would like to see an example of where the NBBO was traded through, including the CQS/CTS logs, here is the link:

Quote from the post:

Basically, the dark pool here should have been fined. (Incidentally, the 'D' that
appears before the '=' sign in the cts entry indicates that the trade did take place
on a dark.) They should either be fined because (1) they traded through the NBBO or
(2) they had a reason to trade through the NBBO and didn't properly report the trade.

I wanted to point this out because there were several references in this thread that said the NBBO was always honored. It is simply not true. . .

EconomicDisconnect said...


Kid Dynamite said...

3:19am: we already have rules that make that illegal, so we don't need more rules to make it more illegal - we just need to enforce the rules we already have.

i can't speak to that stream of jibberish on the post, you can actually see that he took out a whole bunch of lines, edited with the comment "bunch of other quotes."

I have never seen a situation where a stock was quoted 19.50-19.60 and a 19.86 trade printed without lifting the 19.60 offer first - it certainly should NOT happen. If it happened, it should certainly be prosecuted.

Kid Dynamite said...

ps, anon 3:19 - what % of the time would you say that the NBBO is honored? 99.99%? 99.96%? my point is only that if we're going to have reasonable discussions about market structure, we need to start with reasonable assumptions. I think "NBBO is honored" is a reasonable assumption, although there will be glitches where violations occur, as you pointed out.

Anonymous said...

It is bad when brokers are selling mid-point execution services when they know that their clients are getting stale mid-point pricing. That is the issue. Of course, that means their clients are more likely to get filled at dis-advantageous prices.

The brokers should just be straight forward and tell investors that their pricing mechanism is delayed to determine execution pricing.

Kid Dynamite said...

anon @ 9:49am - valid point, and that's a different issue!

Kid Dynamite said...

anon 3:19 - feedback from a colleague re: the post:

" he's right that the print is outside the bbo and therefore if was a regular way print, would be an nms violation. I don't think he's right, though that a D condition code means "dark pool". Dark pool and all broker-dealer "off exchange" (cap commit for example) report to the TRF (trade repoting facility). Whether they are a "dark pool" ats or a BD committing capital to a customer is indistinguishable. To my knowledge there is no "dark pool" flag. D likely means something else (which may have exempted it from nms protection). "

Anonymous said...

Anon @ 9:49 am got it right. That is exactly the trade that was described in the article. Buy was placed in a darkpool for mid point execution. And the execution was done at a stale mid-point.

Please read the story again, KD. And, please consider making your "valid point" comment from 9:59 more prominently displayed on your blog once you see that the 9:49 post was on the money.

Kid Dynamite said...

i don't get your snipe, Anon.

Professional investors who are trading in dark pools which use inferior latency feeds are owed something by someone else? is that what you're suggesting?

if your dark pool's latency is resulting in fills based on 50 millisecond old bid/ask spreads, then use another dark pool. or don't peg to midpoint! it's not quite the smoking gun you seem to think it is.

remember - we're talking about protecting retail investors here - who do not use midpoint pegs in dark pools.

traders using inferior algorithms (which is what these midpoint pegs based on slow feeds are) should look within before looking to blame other traders (latency arb guys) who are taking advantage of their inadequacies.

anyway, anon - if you want to write another post about how midpoint peg algos suck and that brokerage houses should be exposed for even offering them - you can do so on your own blog - it's a different topic, as I explained in my first reply to anon 9:49am.

Kid Dynamite said...

ps anon - in case you're not aware - a midpoint peg order is a limit order. it's a limit order that gets canceled and replaced to a new limit when the midpoint changes. if the limit doesn't get changed fast enough, who's fault is that? if you don't like it, don't use the dark pool. it's NOT the latency arb guy's fault!

i don't know how this comment thread got so off topic. the topic is very simple, and not really debatable, as I laid out in the original post. It's not my opinion, it's a fact that you can't legislate away someone getting the market data first. ban colocation, and everyone will try to put their computers next door to the exchange.. etc etc etc..

more importantly, all of this has basically zero impact on the poor ignorant retail investor who has his limit buy and sell orders sitting out there on the order book. None.

Steve said...

my point about sub-pennying undermining nbbo is that not all mkt participants can trade in sub-penny increments. Thus, I can't really "join" the bid or offer. I can get close, but the real bid and offer are inside me. I've actually given up on posting liquidity and now use marketable limit orders.

Anyway, here's something interesting on HFT and getting data first. It's via Naked Cap and Rajiv Sethi. I think you'll find it intersting assuming you havent' already seen it.

Kid Dynamite said...

absolutely steve - but think about what you're saying... you can't trade in SUB PENNY increments!!! boo hoo! (i mean this in a nice way!) that's a TERRIFIC problem to have, isn't it? that bid ask spreads are SO TIGHT that guys are seeing an advantage by offering 1/2 penny lower than you!

See, this is exactly what i mean when i say that the small investor has never had it better: you don't NEED to post liquidity to avoid paying a 25c bid/ask spread like you did ten years ago - just smack the bid or lift the offer, pay your penny spread, and be happy!

i do the same thing as you- marketable limit orders.

Daniel said...

Sorry to be so late to weigh in, I was in Vegas donking off my trading capital in a WSOP NL tournament...

PeterPeter: "Now, I do feel for the average lay-person who is socking away some money into a 401K that in turn is allocated to a bunch of mutual funds (and may not even realize that they aren't investing). Those people really should not have money in the stock market, but instead in savings accounts with positivie real yield and the bond market. That however is a topic well beyond the scope of KD's point about data access.

I recently put $25 K in a savings account in Wells Fargo and they paid me $2.46 in interest for one month. And banks say they want deposits. Stumpf gets $20 large in comp for '09, and I'm so happy for him, and I get two bucks. Can you say joke? Fixed income for retirement? Seriously? What positive real yield? From where? People, even though they aren't suited for it, to have any hope of a retirement, have to have return from the stock market. There isn't an alternative. As I have said before, I don't have a solution.

Kid Dynamite said...

danny wrote "People, even though they aren't suited for it, to have any hope of a retirement, have to have return from the stock market"

ahhh. yes - another topic entirely, and one probably easily capable of taking up 10,000 words here... the fact that people think they can put their money "in the market" and take out 8%, 10%, 12% or whatever they expect returns with relatively low risk over a 10, 20, or even 30 year period is a HUGE problem...

BUT, again, one of my key points is that anyone looking to attain "market" returns, can do so now easier (via an array of ETFs which allow easy portfolio customization and diversification) and cheaper (lower commish and spread) than ever.

Anonymous said...


Do you also support "quote stuffing?" I mean, if you wanted it bad enough, anyone could send in a trillion orders in a second to overwhelm the markets to create an arbitrage opportunity.


Kid Dynamite said...

Anon - 090410:

i don't see any benefit to the market from "quote stuffing," and I'm not going to defend it, but I would urge you not to believe everything you read on ZeroHedge.

QuoteStuffing, while not helpful, is not nearly as harmful or conspiratorial as they'd have you believe. I don't want to get into it here, but the bottom line is that the theory that you can slow down your competitors by flooding them with quotes they have to process is bunk - you slow yourself down.

And no, I don't think you can create an arbitrage opportunity - talk to some HFT practitioners about this.