Intro

So I've been lurking here for a while now, but just started to dip my feet into this stuff the past few weeks. When I was doing my initial research about what sort of software I would need to get up and running, many people suggested prebuilt trading pipelines such as Quantopian.

For a few days, I was struggling to find "just the right amount of wheel to reinvent", if that makes sense. I am a linux user and greatly enjoy setting most things up from scratch, from my text editor to my music player. I want to know how everything works, and not rely on the whims of an external entity. Relying on a prebuilt set of proprietary tools immediately went against my gut instinct, and the recent closure of Quantopian has only reaffirmed this.

Now I am realizing that I should be rolling most, if not all, of my infrastructure from scratch. So far, I have been getting myself familiar with pulling down data, processing it with a basic SMAC or MACD strategy, and plotting some basic metrics with matplotlib.

Now obviously I have a lot of work to do when it comes to actually developing a deployable strategy, but I would like to lay the software groundwork to make that as painless as possible. I'm not really even worried about deployment because that's a ways off for me.

Discussion

So my question is this: How is your general infrastructure laid out?

• Do you keep your backtesting and live trading separated? I envision backtesting being used to narrow down my strategy, which then I could plug into a driver program that actually executes trades.
• What is your workflow for adding/developing a new strategy?
• How much of your code is modular/reusable?

Currently, I have each strategy separated into its own file, and then a tiny driver program that prompts me for a symbol and runs it through the strategy. A coupe key metrics are calculated, but i'm really just experimenting.

Ideally, I would like to have modular, reusable code that will allow me to develop strategies quickly and give me an idea of their performance through backtesting. I think that passing pandas dataframes from program to program makes sense as a "universal" data type. For example I could have a maximum_drawdown.py that I can call from any strategy, passing in a dataframe of returns.

• What key metrics should I be looking at?
• Do you stick with a few symbols/currency pairs or dynamically search for them?
• Do you backtest with a papertrading account (keeping track of portfolio) or just by the raw data?
• Do you look for a strategy that works ok on lots of symbols or really well on one?

Any ideas or input would be appreciated. Thanks!