| StockFetcher Forums · Filter Exchange · HOW TO DESIGN A SYSTEM (NOT JUST A FILTER) | << 1 ... 39 40 41 42 43 >>Post Follow-up |
| amtmail 34 posts msg #117332 - Ignore amtmail |
12/12/2013 1:04:58 AM when i back test the system i put (select by RSI(2) ascending), is this correct or it should be( select by volume descending) ? |
| Kevin_in_GA 4,599 posts msg #117333 - Ignore Kevin_in_GA |
12/12/2013 8:41:00 AM No - I already answered this question earlier in the thread. You sort by zscore(16) but you need to put the following in as well: Here is the specific code I use in the backtest for sorting: SET{PRICERATIO, CLOSE / IND(^spx,CLOSE)} SET{RATIOMA16, CMA(PRICERATIO,16)} SET{RATIOSTD16, CSTDDEV(PRICERATIO,16)} SET{DIFF16, PRICERATIO - RATIOMA16} SET{ZSCORE16, DIFF16 / RATIOSTD16} sort by zscore16 ascending |
| Kegger 3 posts msg #118752 - Ignore Kegger |
3/25/2014 2:04:29 PM Can someone help me a little? I must be doing something wrong. I backtested this filter (the original posted filter) from the period of 1/1/2002 - 1/1/2004 and started with a initial account of 100,000 and ended with an account balance of 721,957! These results are wildly different than the original results so clearly I'm missing something. Here are the exact settings I used: Entry filter: /*FIRST DETERMINE HISTORICAL RATIO OF S&P STOCK TO THE SPY OVER THE LAST 16 DAYS*/ SET{PRICERATIO, CLOSE / IND(^SPX,CLOSE)} SET{RATIOMA16, CMA(PRICERATIO,16)} SET{RATIOSTD16, CSTDDEV(PRICERATIO,16)} SET{DIFF16, PRICERATIO - RATIOMA16} SET{ZSCORE16, DIFF16 / RATIOSTD16} SET{THRESHOLD16, RATIOSTD16 * 2} /*NEXT, SET CRITERIA NECESSARY TO TRIGGER A PAIR TRADE*/ SET{UPPERBAND16, RATIOMA16 + THRESHOLD16} SET{LOWERBAND16, RATIOMA16 - THRESHOLD16} zscore16 below -2 williams %R(16) below -94 close below lower Bollinger Band(16,2) close above MA(200) Exit Criteria: SET{PRICERATIO, CLOSE / IND(^SPX,CLOSE)} SET{RATIOMA16, CMA(PRICERATIO,16)} SET{RATIOSTD16, CSTDDEV(PRICERATIO,16)} SET{DIFF16, PRICERATIO - RATIOMA16} SET{ZSCORE16, DIFF16 / RATIOSTD16} SET{THRESHOLD16, RATIOSTD16 * 2} zscore16 above -1 Selection Method: SET{PRICERATIO, CLOSE / IND(^SPX,CLOSE)} SET{RATIOMA16, CMA(PRICERATIO,16)} SET{RATIOSTD16, CSTDDEV(PRICERATIO,16)} SET{DIFF16, PRICERATIO - RATIOMA16} SET{ZSCORE16, DIFF16 / RATIOSTD16} select by ZSCORE16 ascending Commission per trade: 5.95 Max Holding Days: 20 Shuffle Entry: No Entry Price: open Conditional Entry: No Use Match Date Close: No Exit Price: open Maximum Trades Per Day: 2 Maximum Open Positions: 5 Maximum Selected Stocks: 10 Close OPEN Positions at End: No Allow Multiple Positions Per Symbol: No Force Same Day Trade: No FWIW - I tried adjusting max trades per day, max open positions, and max holding days and they affected the results down a little, but still I was ending around 600-700K everytime. The original results from Kevin showed much different results for this time period. I tried looking through other posts but this thread is extremely long so apologies in advance if this has already been asked before. |
| Kevin_in_GA 4,599 posts msg #118755 - Ignore Kevin_in_GA |
3/25/2014 6:38:18 PM Maximum Trades Per Day: 2 Maximum Open Positions: 5 Maximum Selected Stocks: 10 +++++++ The original equity curve posted was using 10 positions, with compounding. All the above variables should be set to 10. However, you are probably going to get a different result as I have not seen a high degree of alignment between SF backtesting and other backtests. Also, I used 8.95 per trade and allowed 0.02 in the bid/ask spread. |
| Kevin_in_GA 4,599 posts msg #118756 - Ignore Kevin_in_GA |
3/25/2014 6:39:36 PM Also, make sure you are only selecting S&P 500 as the market (the very first line in the original filter). |
| Kegger 3 posts msg #118768 - Ignore Kegger |
3/26/2014 10:02:17 AM Ah ha! I missed that very first line! What huge difference that made. Using those settings the numbers are a lot more in line with what I expected to see. Also, I didn't see an option in SF to control the bid/ask spread, I'm guessing you tested this somewhere else. Anyway, thanks for the help and the quick response! Of course, this all begs the question, why were my returns so much higher? I understand now that I was correlating stocks to an index that they don't belong to, probably ruining the entire basis of the pair trade that this system was built on. So is there any value to exploring this or was it just dumb luck that the returns were ridiculous? I guess there is only one way to find out....more testing! |
| Kevin_in_GA 4,599 posts msg #118775 - Ignore Kevin_in_GA |
3/26/2014 11:51:12 AM One issue you are encountering is survivorship bias - the stocks that SF lists today do not include all of the stocks that were traded in 2002. Many, many delisted stocks that are not reflected in your backtesting, especially among small and microcaps. In your backtesting you are unintentionally selecting only those which have survived and prospered (this usually leads to better results than you should have gotten). My suggestion is to use a more recent timeframe. I develop and optimize my systems using 1/2/2007 - 1/2/2012 since that contains a massive drop as well as a huge recovery. Any good system should be stable to lots of different market conditions. Then test the system against the out-of-sample period 1/2/2012 - 1/2/2014 and see how well it did. This gives you a better idea of how the system handles new data and is more reflective of how it might trade going forward. I also use the S&P 500 since the list does not turn over all that fast, thus minimizing survivorship bias. |
| frsrblch 35 posts msg #118776 - Ignore frsrblch |
3/26/2014 12:15:50 PM Usually when I get massive returns on a backtest, I've made an error somewhere in my Excel calculations, but if the SF backtest is spitting out these results, I would say this warrants further study. How did this broader filter perform over more recent periods, and how does it compare to the S&P500-only version of the filter? |
| Kegger 3 posts msg #118785 - Ignore Kegger |
3/26/2014 10:11:20 PM So let me start with the baseline filter results, adding in the missing "S&P 500" line to select only s&p 500 stocks. Here are the numbers for the last 12 years, in 2 year increments. Each 2 years starts over with 100K initial balance. It was a little easier than trying to compound the results and it lended for some nice comparisons as well. These are my results for the original filter: Period Ending Value 2002-2004 216,879 2004-2006 137,264 2006-2008 120,913 2008-2010 158,622 2010-2012 142,129 2012-2014 172,025 Then, I removed the "S&P 500" entry criteria and tested again: Period Ending Value 2002-2004 721,957 2004-2006 233,169 2006-2008 467,942 2008-2010 209,871 2010-2012 2,538,315 2012-2014 630,692 Yeah, thats ridiculous. 100K to 2.5M in 2 years. haha fat chance. I dug into that item a little deeper and noticed that most of the stocks were penny stocks, which not only would be insane to trade at that volume it probably wouldn't even be feasible. So I added a limit to the entry criteria of the next test of "price above 1": Period Ending Value 2002-2004 686,574 2004-2006 275,781 2006-2008 229,521 2008-2010 195,050 2010-2012 481,360 2012-2014 265,701 Still pretty insane, and thinking too good to be true. Deeper dive showed a lot of OTC stocks, which I can't trade with my account anyway normally. So I'll filter those out next by adding "market is not otcbb" to the entry filter: Period Ending Value 2002-2004 662,977 2004-2006 175,977 2006-2008 291,727 2008-2010 113,527 2010-2012 186,958 2012-2014 253,748 So now its looking like something that might actually be tradeable for me. Except when I check my account I notice that I get charged more per trade depending on volume. So normally I get charged 5.95/trade, but if I trade 1500 shares it goes up to 8.95, and if I try to trade like 4000 shares now I'm paying $20/trade. Is that normal? Anyway, its what I have to work with. I could test that by breaking up into sections by price but now I've tripled or quadrupled my testing effort. So I take the cowards way out and just limit the low end to stocks above $5 with "price above 5 and market is not otcbb": Period Ending Value 2002-2004 247,581 2004-2006 168,242 2006-2008 189,716 2008-2010 106,933 2010-2012 126,807 2012-2014 186,189 So now I'm back to something I could put into real use. Comparing to the original filter results, I see that this version does better in good years but worse in bad years. I suppose I should compound the results to verify, but I suspect this new version does perform better overall. So where does that leave me? I guess the underlying assumption that pair trading has to be correlated to the index is still mostly valid, but maybe it doesn't really matter which index is used. I should try the same with a different index and see what happens. Anyway, thanks for the great initial post on this, it really got me thinking in different directions and I feel like I'm a better trader for it. |
| novacane32000 331 posts msg #118835 - Ignore novacane32000 |
3/28/2014 8:23:13 PM Drawdowns are the first thing I look at when backtesting. Doesnt matter if you are making 40% a yr for 10 yrs straight,if you have a 50% drawdown in just one of those yrs that is enough to scare me off. Could you stomach losing half your acct? Trade for modest and consistent profits and protect your hard earned money. Trying to get rich off trading will make you poor. |
| StockFetcher Forums · Filter Exchange · HOW TO DESIGN A SYSTEM (NOT JUST A FILTER) | << 1 ... 39 40 41 42 43 >>Post Follow-up |
Copyright 2026 - Vestyl Software L.L.C.•Terms of Service | License | Questions or comments? Contact Us
EOD Data sources: DDFPlus & CSI Data
Quotes delayed during active market hours. Delay times are at least 15 mins for NASDAQ, 20 mins for NYSE and Amex. Delayed intraday data provided by DDFPlus