Saturday, July 27, 2013

A Rant: On Intelligence, Emergent Phenomena, and Alpha


-          Social contracts can be viewed as a balance between the individual and the collective

-          This balance is reevaluated as technology allows for changes in the number, strength, and speed of connections between individuals

-          As technology has enabled the number and speed of connections to be greater than ever before, reaction time of social structures is faster and more significant than that of the individual

-          This has already put significant strain on our legal systems, which struggle to adapt

-          It is likely that concepts such as individual privacy and individual rights will be redefined as these structures exert their increasing influence on society

-          Interactions of these emergent entities are predictable; it is possible to distil signal from the sea of noise


1. The Arrogance of Individualism

Previously, I wrote:

"The difficulty folks seem to have in looking beyond the absurd assumption of the finality of individualism in constructing most -ism's - as opposed to seeing individualism as a product of the proto-industrial forces that shaped the 12th century - is something I consider amazing."

The following is an attempt to clarify my position.


2. Fish searching for the name of water

2.1 A romp through history

At the dawn of humanity, tribes, bound by a shared creation myth, found a balance between the individual and the collective. As individuals competed within a tribe, so tribes competed, with tribes that balanced the interests of the tribe with those of the individual best emerging from the Darwinian struggle.[1]

The nature of this balance changed over time. Painting with an extremely broad brush, the nature of the balance between the individual and the collective changed with the coming of agriculture, where nests and ancestral homes became central to defend, culminating in the middle ages, reaching a critical juncture during the age of monarchy. As writing, then printing slowly permeated societies, the age of kings gave way to the age of enlightenment, ushering in a renewed social contract. The individual continued to gain prominence as labor-saving devices laid the seeds of the industrial revolution. With time, the corporation, a pale reflection of the individual, was conceived: a caricature of the individual that birthed it, it nursed the industrial revolution into maturity.

2.2. Dynamic -isms

With these changes came methods of organization, which I refer to as "isms". Modern isms are neatly tabulated by Ebenstein and Fogleman in _Todays Isms: Socialism, Capitalism, Fascism, Communism, Libertarianism_. By their very nature, these labels sought to organize social structures that existed at that point in time. To illustrate, the concept of Libertarianism would have very little meaning in ancient Egypt; similarly, capitalism was viciously punished in medieval Europe, as discussed in Heilbroner's _The Worldly Philosophers_.

During most of this time, we considered our social structures sacrosanct, our conceptions of societal roles and structures complete. Each year, each decade, each generation, each age, we were recurrently surprised as they fell apart, and new norms, new societal contracts, new balances between the individual and the collective emerged.

Whether we smashed power looms in the name of General Ludd or applauded in corporate technological advances, whether we decried proclamations of God's death or embraced the proclamation of promised freedom, the new balance found quiescence for a few months, years, generations. Then, the seed germinated and blossomed into the sensibilities of the next brief period of quiescent development.

And so, two thousand centuries after the dawn of Homo sapiens, we find ourselves swimming in the zeitgeist of Pax Americana, amidst change that is (we are assured) faster than ever before in human history. [2] We are more connected than ever before, and alleviate every limitation we can using technology. Information about each of us is more available than ever before, and is harvested, poked, marketed to, and stored in giant underground datacenters.

Kurzweil's cries for singularity echo in our zeitgeist as quietly as Paul Reveres' whispered warnings[3]: "The regulars are coming," Kurzweil cries. They'll be here any minute now. Get ready.

Meanwhile, social change is sparked throughout the cradle of civilization, facilitated by an Arab spring that erupted in the same region where, millennia ago, Moses struck water from a rock, creating a spring that would reverberate over four thousand years.

We watch enraptured, not seeing the forest of the trees. What does it all mean? We struggle to connect the dots, certain that there is a pattern, some meaning. We strain, as goldfish in a bowl, struggling in an attempt to name water.

The problem, of course, is that "What does it all mean? is a patently terrible question.[42] Perhaps a better question is, "Where are we going? What's next?"


3. Emergent Phenomenon: On Layers of Complexity

3.1. On Sentience

To consider the question and attempt to name the zeitgeist we swim in daily, let us step back and consider what self-aware sentience we so flippantly label "human" entails.

Hofstadter's anthill

Consider the humble anthill. Assuming we knew how to "read" the pattern of ants, we would be able to see changes in activity when rain was expected; when the colony was low on food; when there was a rival colony nearby. This is analogous to being able to "listen to" the anthill--the examples here roughly translate to "It's going to rain," "I'm hungry," and "Stranger danger!"

Assuming we did have this skill, our communication with the anthill would still be quite limited. There's a good chance that we wouldn't have much to say to it, and that its ("her?") topics of conversations are very likely going to be quite boring.

But then, an anthill is quite a simple construct. How, then, do we go about deciding if something is complex enough to be an interesting conversationalist, meriting perhaps a dinner invitation?

3.2. Picking Interesting Dinner Companions

Information Integration Theory of Consciousness

One interesting approach is the one taken by Koch, Griffith, et al, that uses the concept of Kolmogorov Complexity to determine how complex a system is. On this basis, the Information Integration Theory of Consciousness (IITC, has been developed, which attempts to quantify how complex intelligence really is.

However, a definition that relies solely on complexity is quite lacking. For example, the average Rube Goldberg device is extremely complex, but one would hardly consider it an interesting dinner companion.

Future Histories

To this end, another dimension to the definition of intelligence is presented by Wissner-Gross and Freer. They focus on the number of "Future Histories" a system is able to account for, and discuss an extremely elegant mathematical treatment in Causal Entropic Forces ( The short paper is likely to be a landmark publication, in addition to being a strong contender for "Most Deceptively Humble Title of the Decade."

Each of these views holds a mirror to sentience, a perspective mapping a projection of the whole. Consider, then, that these characteristics can be applied quite easily to other, higher, levels of organization. What, then, stops these higher levels of organization from being considered sentient?

4. The Borg: you really think it will be this obvious?

Kurzweil has, for years now, predicted the singularity. "They're coming," he promises, "and we have no way of knowing what will happen when they get here."

Perhaps there is a singularity on the horizon, a major phase-shift that will redefine the human race in its entirety. However, I posit that "they're coming" is inaccurate.

They're here.

4.1 Dramatis Personae

At this point, I would like to introduce a colorful cast of characters. I invite you to picture them in your mind as you read their description, and try not to chuckle.

-          Twitter: The overly caffeinated Manic ADHD

-          LinkedIn: Three-piece empty suit waving list of bullet-point accomplishments.

-          Facebook: People you see every thanksgiving, the ones that remind you that we don't pick our relatives.

-          Reddit: Mid-twenties hipster, well-read, informed, armchair intellectual

-          Quora: Introverted nerd

-          Anonymous: Discordia, Dionysian reverberation of Guy Fawkes and Gavrilo Princip

-          Intelligence Community Comprehensive National Cybersecurity Initiative: Agent Smith

-          Usenet (circa 1992): A herd of performing elephants with diarrhea--massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it

4.2. O RLY?

Of course, the same criteria can be applied to much older structures. What of the church? What of the state? Or the humble anthill, aforementioned? Aren't they the same thing?

Nope. Considering the criteria, glaring differences lie in degrees of effectiveness and reaction times.

Anonymous can devastate armies more comprehensively than Guy Fawkes or Gavrilo Princip could imagine; Government datacenters are more informed, aware, and capable than any warmachine Genghis Khan, Tamerlane, Alexander the Great, or Papal Rome could ever envision.

More importantly, for the first time in history, our institutions can react at the speed of thought. Anonymous can act as fast as any human; twitter responds minutes, if not hours, faster than traditional news source; Facebook ruins reputations faster than an inebriated weekend in Las Vegas.

Finally, institutions are created, live, evolve, and die, faster than ever before. They spontaneously coalesce, act, and, robbed of energy, dissipate, as dust-devils. While the UseNet reference in the list above no doubt brings a wry smile to the few readers who have braved my clumsy prose this far, how strange would it have been to see MySpace in that list?

4.3. YA RLY

The impact of these personalities being born, living, interacting, and, eventually, dying, in our zeitgeist is slowly asserting itself. Our legal system, so focused on the individual, or on explicitly declared organizations such as corporations, creaks and groans, struggling--and, frankly, failing--to adapt. Our notions of individual privacy, and, increasingly, individual rights, seem increasingly quaint, bordering on ironic, and fade to irrelevance.

We search for a new quiescent point, looking for that next stable point to reorient our sensibilities amid familiar structures. But who knows what rough beast, its hour come round at last, slouches towards Bethlehem to be born?


5. Stay Thirsty, My Friends

Contrary to the popular understanding, emergent structures at this level are far from unpredictable. Mandelbrot was the tip of the spear, leaving us a legacy that reads as a survey of an idea whose time has come. Stogatz, Kuramoto, Crawford how us how to analyze pieces of a much larger system, adding to our toolbox. Sornette tames the chaotic wilds to some degree, offering some refuge in a framework to build digital oracles that espouse pythy predictions.

Taleb teaches us to ensure that any action we take based on our predictions is built on an antifragile foundation. Thales of Miletus tells us that we must act.

As the Jotun, Aesir, and Vanir of our age scheme, plot, live, love, and die in our ether, we can listen in. There is an abundance of signal in the apparent noise, and I am going to find it. There's alpha here. Stay thirsty, my friends.



[1] Wilson covers this best in _The Social Conquest of Earth_. I have copied a snippet below.


Since Paleolithic times each tribe--of which there have been countless thousand--invented its own creation myth. During this long dreamtime of our ancestors, supernatural beings spoke to shamans and prophets. They identified themselves to the mortals variously as God, a tribe of Gods, a divine family, the Great Spirit, the Sun, ghosts of forebears, supreme serpents, hybrids of sundry animals, chimeras of men and beasts, omnipotent sky spiders--anything, everything that could be conjured by dreams, hallucinogens, and fertile imaginations of the spiritual leaders. They were shaped in part by the environments of those who invented them. In Polynesia, gods pried the sky apart from the ground and sea, and the creation of life and humanity followed. In the desert-dwelling patriarchies of Judaism, Christianity, and Islam, prophets conceived, not surprisingly, a divine, all-powerful patriarch who speaks to his people through sacred scripture.

The creation stories gave the members of each tribe an explanation for their existence. It made them feel loved and protected above all other tribes. In return, gods demanded absolute belief and obedience. And rightly so. The creation myth was the essential bond that held the tribe together. It provided believers with a unique identity, commanded their fidelity, strengthened order, vouchsafed law, encouraged valor and sacrifice, and offered meaning to the cycles of life and death. No tribe could long survive without the meaning of its existence defined by a creation story. The option was to weaken, dissolve, and die. In the early history of each tribe, the myth therefore had to be set in stone.

The creation myth is a Darwinian device for survival.


[2] I remain unconvinced that we are changing faster than ever, but have tabled investigation in that area for now.


[3] - Paul Revere really did whisper the warnings, as secrecy was paramount to his mission.

Tuesday, June 18, 2013

On The Inevitability of Big Brother

I accidentally committed (pun intended) a blog post when commenting on a friend's social media widget. I thought I would capture it here. 
The magnitude of the outrage over the revelations of the PRISM program caught quite a few off guard, I'm sure. Especially since this is not a new narrative. It has waxed and waned since the Renaissance at least, appearing in the works of Bacon, Machiavelli, Nietzsche (tangentially), Hegel, to list just a few.[0]

The narrative gained its stride during the industrial revolution, as the world progressed through the birth pangs of the practical implications concepts such as those optimistically, if irreverently, labeled "self determinism."[1]

Not saying both dystopias don't exist, but... well, this comic says it best. No need to reiterate.

To summarize, while worrying about the Orwellian vision is appropriate, I wouldn't want to ignore Huxley's prophecies.

Additionally, if you have the supply of Advil required, reading Das Kapital and Contribution to the Critique, stripped of political polemic predicts the path of decades since with nontrivial accuracy. (ref. Marx's dialectical analysis and 6 stages.) [2]

This rant bought to you with my nine-month long obsession with complexity theory, and recent discovery of Dr. Didier Sornette's work with feed-forward systems aggregated over subjective probability ensembles. I'm a fan.

TED talk:

[0] I shudder to think how many orders of magnitude more sources an informed person - a Taleb (pun intended) - could name.

[1] I do wonder how much it was fueled by disillusionment that was a reaction to being caught between the reverberations of the Great Wars and the emerging zeitgeist that seemed (I'm guessing) ideological, and, prima facie, iconoclastic.

[2] I do disagree with Marx on the specifics. I further disagree with him in his sentiment that we stop at #6. The difficulty folks seem to have in looking beyond the absurd assumption of the finality of individualism in constructing most -ism's - as opposed to seeing individualism as a product of the proto-industrial forces that shaped the 12th century - is something I consider amazing. It's a strange blind spot. But that's a tangential rant for some other time.

Saturday, June 16, 2012

On Student Loans

“I could end the deficit in five minutes. You just pass a law that says that any time there's a deficit of more than three percent of GDP, all sitting members of Congress are ineligible for re-election.”
                                - Warren Buffet

“As long as the music is playing, you’ve got to get up and dance, we’re still dancing.”
                                 - Chuck Prince, July 2007, as CEO of Citigroup

Recently, there has been much talk of a student loan debt “bubble,” record defaults, and associated predictions of an oncoming major dislocation in the education sector. As always, when faced with industry-level talk of this nature, I ask myself: “What’s the trade?”

Short Sally Mae, the entity on the hook for all these loans? What about Apollo Group and its associated cohort? Too obvious. Additionally, APOL actually has a 10% short float already, so I’m hardly the first to see this trade.

Finally, I’ve never been very good at playing these “imminent dislocations:” while we revere those who made their careers shorting the housing market in 2007 (a la Lewis’ The Big Short), we forget the legions of trampled bears who could not stay solvent long enough. In our current environment, there are greater macro trends in play, and lacking a catalyst, I see no reason to take the short trade.

Instead, the optimist in me tells me to wait. Sometime soon, the obvious solution to the student loan issue, and possibly several social situations commonly perceived as related, should present itself. Here, then, is what I’ll be looking for.

It seems a little strange that student loans tend to have similar rates, regardless of major or school choice. I suspect it would take an actuary, armed with the appropriate data, just a few hours to come up with risk of default metrics based on school and major. These loans can then be offered at their respective rates, based on what school the student is going to, what their major is, and any other creative factors said actuary could distill. Eventually, those making the loans can also sell the related paper, providing yet another source of revenue.

Of course, the forward-feedback loop this will create will incentivise certain majors and schools -- those that historically repay loans -- and disincentivize others. Tangentially, this will likely draw flak from people who have strong opinions of what the world “should” look like, regardless of what the world actually looks like. I can’t help but wonder how many of these critiques will come from the same folks that bemoan the current structure of student loans.

This, then, is the trade. I’m going to start looking around for something that resembles the structure described above. When it presents itself, I plan on going long. How do I know it will? I consider it inevitable: the current structure is unsustainable, and I haven’t heard any better ideas.

...or am I just missing something big and obvious? Won’t be the first time.

Sunday, April 1, 2012

Frequency, Not Intensity: Going Slower to Arrive Faster

“If hunger is a problem in America, then why do we have an obesity problem among the people who we say have a hunger program?                                                                                                                                                  -Rick Santorum
One day, I woke up way too fat because I was too poor to afford food, and far too preoccupied with that fact to know better. At 5’11”, I stopped measuring when the scale hit 250lbs. There didn’t seem to be any point.

Fast forward three years, and I had completed a marathon, could do 20 pull-ups easily, and ran 3 miles in just under 17 minutes. While not exactly easy on the eyes -- sadly, you can’t sweat out ugly -- at 165lbs, I was no longer immediately repulsive to my own mother. I had managed to lose a third of a me.

I had replaced said third-of-a-me with a nasty limp, various lower-back injuries, and a tendency to crackle while breathing (lung injuries are, at the very least, entertaining). But damnit, every morning, I was going to run my 5k. After work, I was getting my quick workouts done, come hell or high water. 



A few measly injuries weren’t going to stop me. Mind over matter! 

...until, of course, matter, being bruised, ripped and mostly sponge, finds itself no longer able to support mind. Tore up as a soup sandwich, I learned that limping up to a treadmill with a walking stick draws stares at the gym.

It was six months of painful, slow healing before I was able to walk normally again, ten months before I ran more than a few yards. My body fat crept back into the double-digits, and my Ironman hopes were set aside for some time.

My story, of course, is far from unique. I had fallen into the same trap so many of us do when we attempt long-lasting change. I had treated obesity like a one-off problem to be solved, a dragon to be slain. Neglecting feedback, mistaking stubbornness for resolve, I had ploughed ahead. That didn’t work, so I doubled it.

About now, this should start sounding familiar to traders.

How many times have we gone after a revenge trade, shorting after a rip with a flimsy story? Does “Yeah, it ripped hard, but now it’s made a doji on the upper bollinger band, it’s got to drop now!” sound familiar? That’s ignoring shin-splints and the knotted muscles and going for another morning run. That’s ignoring feedback.

How many times have we, in an attempt to make up for a red day, gone in with double or triple weight? If you’re having a red day or week or quarter, it’s probably because your read isn’t sighted in. It ain’t workin’. Don’t double it.

All of these mistakes come, in some part, from a short-term thought process. Like my (doomed) attempt at a 125lbs clean-and-jerk with no warm-up, they’re looking to take something that should be long-term and achieve it in the short-term. The way I see it, the only thing a short-term thought process leads to is a short-term trading career. 

As a trader, a plan for doing this for the rest of your life translates to managing risk in day-to-day operations. After all, if you don’t manage risk, say by doubling down when things aren’t working, then you reduce your chances -- exponentially, in fact -- of sticking around.

In the words of my co-worker and workout partner, on seeing me stumble into the gym: “You want to be in a position where you’re doing this for the rest of your life. You can barely stand. Take the damn day off.” We’re in this for the long haul. Optimize for frequency, not intensity. Manage risk: size trades appropriately, take cushion early, respect stops.

The alternative is to ignore feedback, and, when faced with a losing course of action, double up and try again. It’s what makes you push well past the boundaries of sense, well past where you’re coughing up red gibbets, keeping the needle firmly buried in the redline. Not surprisingly, the results will be similar. Like my lower back, the account will blow up. Much like me and my marathons, you’ll sideline yourself for months, maybe years, and perhaps even permanently.

Having realized that I had been confusing intensity for frequency, I have not rushed back into my training with the same reckless abandon. I am still running, thankfully, but rarely do I try for those 6-minute mile runs day after day after day. And while my 5k times are improving, I am still far, far away from that 17 minute mark. I am optimizing for frequency first and intensity second. I plan on taking it a little slower this time.

Funny thing is, if I had taken it slower the first time around, I would’ve arrived a whole lot faster.

Tuesday, March 20, 2012

A Quick Detour

The first part of trade automation that I am going to discuss is trade identification. As this involves both identification and a quick risk/reward analysis, before we get too far, I wanted to introduce a quick and dirty way to get a feel for targets and stops.

One very quick way to do this is to have pre-determined trading levels. For example, intra-day, we can use daily and weekly pivots. On longer-term timeframes, we can identify support and resistance levels on charts.

Using python, I quickly whipped up some support/resistance level using the most basic of algorithms. I defined support or resistance as that point where d(Price)/dt = 0, using close-only values to keep things simple. Other additions such as volume weighting, price binning, and intra-day movement can be added quickly to give better levels.

The results:

In the next post, I will use this simple chart to discuss risk/reward and trade identification for an automated trading system.

Thursday, March 1, 2012

On Automation: The First Steps

If you follow me on Stocktwits or twitter, you most likely know that I run a number of systems that generate my trades. These have produced excellent results over the past two years, and I am finally taking the inevitable step of automating them. As my work may help someone, I will blog about the process as I go through it.

I should start by saying that trading systems, especially automated ones,  are far from "silver bullets." All they do generate an "edge," similar to the edge casinos have built into their games. This is described by Mark Douglas in Trading in the Zone, and he delves into the concept far better than I have the ability to.

One of Douglas' little gems is that the nature of the edge is immaterial, so long as the edge is statistically valid. My personal experience watching traders of all types over the years has borne this out. Consequently, I will not discuss the specifics of my trading systems. Frankly, that is probably a good thing: the system is the least interesting part.

I will focus, instead, on the process of automation. As an overview, I will roughly consider the following categories over the next few posts:
  • Trade identification: Indicators, trend, other inputs such as market events
  • Trade execution: Entry, Target, Stop, Timeout, other details of how the rubber meets the road
  • Risk management: Asset allocation per trade, increasing/decreasing allocation, etc.
  • Implementation: Platform selection back-testing statistics, and other mundane matters
Due to my personal preferences, I have focused on trading options using these automated systems. However, I believe futures represent a preferable cost-structure (possibly at the cost of truly terrifying leverage), and so I will also discuss trading /ES and /TF.

I plan on keeping this as a higher-level discussion, and not devolve into a step-by-step how-to. As always, of course, I remain available to answer questions via twitter.

Tuesday, February 28, 2012

Updated Confusion

When we last looked over the macro picture, I presented bonds at the bottom of a descending channel. Let us get a quick update on the bond situation:

As hoped, bonds have bounced and are now testing the top of the descending channel. Despite this upwards push, the market has continued to fair well, with dips being very, very shallow and aggressively bought so far. We have since pushed from SPX 1330s to 1370, a level that many are watching, and one that is proving to be quite stubborn.

While a selloff in the bnods would be bullish for equities and allow us to push through the bother of SPX 1370, the US Dollar is telling a slightly different story, via the UUP.

The UUP is resting on the 200 SMA on the daily, and the 50 SMA on the weekly. While the correlation between the dollar and the market has loosened recently, this kind of support may cause the USD to rip enough to make a difference.

If the DX, and correspondingly, the UUP, lose these levels, then I think we should strap in for a trip to the 1400s. For now, however, I continue long. And although I am scared, I'm leaving it to my stops tell me when I'm wrong.