Monday, 26 February 2018

Meucci MOOC and Marathon

A quik update on what's new at Attilio Meucci's ARPM course offerngs in 2018.

  • ARPM MOOC : a light package for self-learners: 25 hours of online instruction, split into 134 micro video lectures.
  • ARPM Marathon : provides a deep absorption of the materials: 120 hours of online instruction, with graded homework and one-on-one discussions.

And of course the original ARPM Bootcamp is still held in person in NYC this year too. Enjoy!

Wednesday, 6 July 2016

mini-Meucci : Applying The Checklist - Steps 10+

In this final leg of The Checklist tour we'll be looking at the Dynamic Allocation step and touch briefly on ex-post Performance Analysis.

Dynamic Allocation

Essentially this involves repeating the previous 9-steps on a periodic basis (e.g. a sequence of monthly allocations) according to a chosen allocation policy.

Examples of dynamic allocations include systematic strategies (based on signals) and portfolio insurance.

See slide #55 Dynamic Allocation (general case).

Quantitative/Systematic Strategies

Those interested in active portfolio management and who attend the ARPM bootcamp, will also have access to the ARPM Lab. In it are 2 very interesting chapters covering both the theory and practice (with code) of quant strategies, where you'll learn among other things, how to construct a characteristic portfolio (Grinold and Easton) based on your signals. For an example of such a characteristic portfolio strategy, see the youtube video on slide #56 Dynamic Allocation (video).

Example Python Code

In our toy example with the goal of constructing a low volatility equity portfolio, our chosen allocation policy will be to weight the 30 DJIA stocks according to the ex-ante minimum variance portfolio, and rebalance the portfolio at the end of each month.

We'll use an expanding historical data window of at least 3 years, apply time-conditioned weights to the observations when estimating the ex-ante distribution, and also use a simple form of shrinkage before optimizing.

To simulate such a sequence of allocations over a 3 year period, we'll use the open source Zipline package.

In [2]:
%matplotlib inline
import rnr_meucci_functions as rnr
import numpy as np
from zipline.api import (set_slippage, slippage, set_commission, commission, 
                         order_target_percent, record, schedule_function,
                         date_rules, time_rules, get_datetime, symbol)

# Set tickers for data loading i.e. DJIA constituents and DIA ETF for benchmark
tickers = ['MMM','AXP','AAPL','BA','CAT','CVX','CSCO','KO','DD','XOM','GE','GS',
           'HD','INTC','IBM','JNJ','JPM','MCD','MRK','MSFT','NKE','PFE','PG',
           'TRV','UNH','UTX','VZ','V','WMT','DIS', 'DIA']

# Set investable asset tickers
asset_tickers = ['MMM','AXP','AAPL','BA','CAT','CVX','CSCO','KO','DD','XOM','GE',
        'GS','HD','INTC','IBM','JNJ','JPM','MCD','MRK','MSFT','NKE','PFE','PG',
        'TRV','UNH','UTX','VZ','V','WMT','DIS']
                         
def initialize(context):
    # Turn off the slippage model
    set_slippage(slippage.FixedSlippage(spread=0.0))
    # Set the commission model
    set_commission(commission.PerShare(cost=0.01, min_trade_cost=1.0))
    context.day = -1 # using zero-based counter for days
    context.set_benchmark(symbol('DIA'))
    context.assets = []
    print('Setup investable assets...')
    for ticker in asset_tickers:
        #print(ticker)
        context.assets.append(symbol(ticker))
    context.n_asset = len(context.assets)
    context.n_portfolio = 40 # num mean-variance efficient portfolios to compute
    context.today = None
    context.tau = None
    context.min_data_window = 756 # min of 3 yrs data for calculations
    context.first_rebal_date = None
    context.first_rebal_idx = None
    context.weights = None
    # Schedule dynamic allocation calcs to occur 1 day before month end - note that
    # actual trading will occur on the close on the last trading day of the month
    schedule_function(rebalance,
                  date_rule=date_rules.month_end(days_offset=1),
                  time_rule=time_rules.market_close())
    # Record some stuff every day
    schedule_function(record_vars,
                  date_rule=date_rules.every_day(),
                  time_rule=time_rules.market_close())

def handle_data(context, data):
    context.day += 1
    #print(context.day)
 
def rebalance(context, data):
    # Wait for 756 trading days (3 yrs) of historical prices before trading
    if context.day < context.min_data_window - 1:
        return
    # Get expanding window of past prices and compute returns
    context.today = get_datetime().date() 
    prices = data.history(context.assets, "price", context.day, "1d")
    if context.first_rebal_date is None:
        context.first_rebal_date = context.today
        context.first_rebal_idx = context.day
        print('Starting dynamic allocation simulation...')
    # Get investment horizon in days ie number of trading days next month
    context.tau = rnr.get_num_days_nxt_month(context.today.month, context.today.year)
    # Calculate HFP distribution
    asset_rets = np.array(prices.pct_change(context.tau).iloc[context.tau:, :])
    num_scenarios = len(asset_rets)
    # Set Flexible Probabilities Using Exponential Smoothing
    half_life_prjn = 252 * 2 # in days
    lambda_prjn = np.log(2) / half_life_prjn
    probs_prjn = np.exp(-lambda_prjn * (np.arange(0, num_scenarios)[::-1]))
    probs_prjn = probs_prjn / sum(probs_prjn)
    mu_pc, sigma2_pc = rnr.fp_mean_cov(asset_rets.T, probs_prjn)
    # Perform shrinkage to mitigate estimation risk
    mu_shrk, sigma2_shrk = rnr.simple_shrinkage(mu_pc, sigma2_pc)
    weights, _, _ = rnr.efficient_frontier_qp_rets(context.n_portfolio, 
                                                          sigma2_shrk, mu_shrk)
    print('Optimal weights calculated 1 day before month end on %s (day=%s)' \
        % (context.today, context.day))
    #print(weights)
    min_var_weights = weights[0,:]
    # Rebalance portfolio accordingly
    for stock, weight in zip(prices.columns, min_var_weights):
        order_target_percent(stock, np.asscalar(weight))
    context.weights = min_var_weights
                      
def record_vars(context, data):
    record(weights=context.weights, tau=context.tau)
        
def analyze(perf, bm_value, start_idx):
    pd.DataFrame({'portfolio':results.portfolio_value,'benchmark':bm_value})\
        .iloc[start_idx:,:].plot(title='Portfolio Performance vs Benchmark',\
        figsize=(10, 8))

if __name__ == '__main__':
    from datetime import datetime
    import pytz
    from zipline.algorithm import TradingAlgorithm
    from zipline.utils.factory import load_bars_from_yahoo
    import pandas as pd
    import matplotlib.pyplot as plt
    
    # Create and run the algorithm.
    algo = TradingAlgorithm(initialize=initialize, handle_data=handle_data)

    start = datetime(2010, 5, 1, 0, 0, 0, 0, pytz.utc)
    end = datetime(2016, 5, 31, 0, 0, 0, 0, pytz.utc)
    print('Getting Yahoo data for 30 DJIA stocks and DIA ETF as benchmark...')
    data = load_bars_from_yahoo(stocks=tickers, start=start, end=end)
    # Check price data
    data.loc[:, :, 'price'].plot(figsize=(8,7), title='Input Price Data')
    plt.ylabel('price in $');
    plt.legend(loc='center left', bbox_to_anchor=(1.0, 0.5))
    plt.show()
    
    # Run algorithm
    results = algo.run(data)
    
    # Fix possible issue with timezone
    results.index = results.index.normalize()
    if results.index.tzinfo is None:
        results.index = results.index.tz_localize('UTC')
    
    # Adjust benchmark returns for delayed trading due to 3 year min data window 
    bm_rets = algo.perf_tracker.all_benchmark_returns
    bm_rets[0:algo.first_rebal_idx + 2] = 0
    bm_rets.name = 'DIA'
    bm_rets.index.freq = None
    bm_value = algo.capital_base * np.cumprod(1+bm_rets)
    
    # Plot portfolio and benchmark values
    analyze(results, bm_value, algo.first_rebal_idx + 1)
    print('End value portfolio = {:.0f}'.format(results.portfolio_value.ix[-1]))
    print('End value benchmark = {:.0f}'.format(bm_value[-1]))
    
    # Plot end weights
    pd.DataFrame(results.weights.ix[-1], index=asset_tickers, columns=['w'])\
        .sort_values('w', ascending=False).plot(kind='bar', \
        title='End Simulation Weights', legend=None, figsize=(10, 8));
Getting Yahoo data for 30 DJIA stocks and DIA ETF as benchmark...
C:\Anaconda3\envs\py34\lib\site-packages\ipykernel\__main__.py:106: DeprecationWarning:
load_bars_from_yahoo is deprecated, please register a yahoo_equities data bundle instead
Setup investable assets...
Starting dynamic allocation simulation...
Optimal weights calculated 1 day before month end on 2013-05-30 (day=774)
Optimal weights calculated 1 day before month end on 2013-06-27 (day=794)
Optimal weights calculated 1 day before month end on 2013-07-30 (day=816)
.
.
.
Optimal weights calculated 1 day before month end on 2016-03-30 (day=1487)
Optimal weights calculated 1 day before month end on 2016-04-28 (day=1508)
Optimal weights calculated 1 day before month end on 2016-05-27 (day=1529)
End value portfolio = 130872
End value benchmark = 125771

Ex-post Performance Analysis

"Ex-post performance analysis is a broad subject that attracts tremendous attention from practitioners, as their compensation is ultimately tied to the results of this analysis. Ex-post performance can be broken down into two components: performance of the target portfolio from the Optimization [Construction] Step P 8 and slippage performance from the Execution Step P 9."
The Prayer (Ex-post Analysis section, former Checklist)

Python Code Example

We'll use the open-source Pyfolio package that works nicely with Zipline, and is quite comprehensive in its analytics.

As you can see below, our toy example of a DJIA stock portfolio was able to achieve lower annualised volatility of 11.74% compared to the DIA ETF of 13.05%. If the volatility is not low enough for you, then you could always expand the universe of stocks or add a negatively correlated bond ETF, like TLT, into the mix - which I'll leave as an exercise for the interested reader.

In [3]:
# Sequel Step - Ex-post performance analysis
import pyfolio as pf

returns, positions, transactions, gross_lev = pf.utils.\
    extract_rets_pos_txn_from_zipline(results)
trade_start = results.index[algo.first_rebal_idx + 1]
trade_end = datetime(2016, 5, 31, 0, 0, 0, 0, pytz.utc)

print('Annualised volatility of the portfolio = {:.4}'.\
    format(pf.timeseries.annual_volatility(returns[trade_start:trade_end])))
print('Annualised volatility of the benchmark = {:.4}'.\
    format(pf.timeseries.annual_volatility(bm_rets[trade_start:trade_end])))
print('')

pf.create_returns_tear_sheet(returns[trade_start:trade_end], 
                             benchmark_rets=bm_rets[trade_start:trade_end])
Annualised volatility of the portfolio = 0.1174
Annualised volatility of the benchmark = 0.1305

Entire data start date: 2013-05-31
Entire data end date: 2016-05-31


Backtest Months: 36
                   Backtest
annual_return          0.09
annual_volatility      0.12
sharpe_ratio           0.82
calmar_ratio           0.77
stability              0.85
max_drawdown          -0.12
omega_ratio            1.15
sortino_ratio          1.19
skewness              -0.04
kurtosis               2.57
information_ratio      0.01
alpha                  0.03
beta                   0.82

[see appendix below for full Pyfolio returns analytics]



That's all folks! Hope you have enjoyed the journey and learnt something along the way...

P.S. If you do sign-up for the Bootcamp, please let the good folks at ARPM know you learnt about it at returnandrisk.com!

References to Attilio Meucci's Work

  1. The Checklist slides
  2. The Prayer (former Checklist)

Download Python Code

Click here for the GitHub repo

Appendix - Full Pyfolio Returns Analytics

Annualised volatility of the portfolio = 0.1174
Annualised volatility of the benchmark = 0.1305

Entire data start date: 2013-05-31
Entire data end date: 2016-05-31


Backtest Months: 36
                   Backtest
annual_return          0.09
annual_volatility      0.12
sharpe_ratio           0.82
calmar_ratio           0.77
stability              0.85
max_drawdown          -0.12
omega_ratio            1.15
sortino_ratio          1.19
skewness              -0.04
kurtosis               2.57
information_ratio      0.01
alpha                  0.03
beta                   0.82

Worst Drawdown Periods
   net drawdown in %  peak date valley date recovery date duration
0              12.23 2015-07-20  2015-08-25    2015-10-23       70
1               6.46 2015-12-29  2016-01-20    2016-02-25       43
4               6.42 2013-08-05  2013-10-08    2013-11-06       68
2               6.39 2014-11-28  2014-12-16    2015-05-19      123
3               6.39 2013-11-29  2014-02-03    2014-03-28       86


2-sigma returns daily    -0.014
2-sigma returns weekly   -0.027
dtype: float64
C:\Anaconda3\envs\py34\lib\site-packages\pyfolio\plotting.py:1356: FutureWarning:
sort(columns=....) is deprecated, use sort_values(by=.....) print(drawdown_df.sort('net drawdown in %', ascending=False))