Evaluating VaR: a qualitative and quantitative impact study

Value-at-Risk (VaR) is the primary measure used to estimate the potential losses for a bank portfolio due to market movements. It has three key roles: internal risk management; risk reporting; determination of market risk regulatory capital for banks with approval for their proprietary regulatory VaR model. These three roles exalt VaR to a position of significant influence. It influences risk allocation and control through the internal risk management role. It influences market perception of a bank’s risk-taking through its reporting role. Through its regulatory capital role, it affects an opportunity cost. VaR has been central to the measurement and management of market risk for banks since its technical development by RiskMetrics in the 1990’s. Having emerged from practice in JP Morgan, it has an atypical genesis as a reporting measure, and unusual longevity not wholly dependent on regulatory imperatives. VaR’s internal control function, its reporting role, and its use in determining market risk regulatory capital means it has the potential to influence risk-taking behaviour of banks with clear societal implications, embodying the interrelationship between accounting and the social (Burchell et al. 1985). VaR’s role within the market risk regulatory capital framework led to its emerging as a cynosure or centre for attention by financial media and was a source of concern in investigations into the events of the 2007-2009 financial crisis. The Basel Committee on Banking Supervision (BCBS) have developed a revised framework for the treatment of market risk, known as the Fundamental Review of the Trading Book (FRTB) with implementation due in January 2022(BCBS 2019). This framework demotes the role of VaR as the central measure in the determination of market risk regulatory capital. The FRTB calculative framework addresses many of the key issues that became apparent in the financial crisis, and includes the replacement of VaR by Expected Shortfall (ES), a measure heralded for capturing tail risk. However, these changes to the market risk regulatory framework do not automatically infer changes to the other roles of VaR. This prompts the need for a qualitative evaluation of VaR within the context of its various roles and its sphere of influence in banking, together with an evaluation of the potential impact of the regulatory changes on risk and portfolio management practice. The necessity to evaluate VaR on a contextual basis prompts consideration to view VaR as an accounting measure, thereby facilitating the use of sociological evaluation methodologies developed for accounting measures. Miller and Power (2013) identify four key roles of accounting measures: territorialising, mediating, adjudicating and subjectivising. This framework has been used to evaluate the performative power and accountability of various applications of accounting (Vosselman 2014; Pelger 2016) and other economised settings including: sustainability (Markota Vukić et al. 2017); environmental impact (Doganova and Karnøe 2015); and marketing knowledge (Jacobi et al. 2015). To the best knowledge of the author, it has so far not been applied to accounting measures that function in risk management within the financial sector, despite the clear societal impact of these technologies. We translate the four key roles of accounting measures identified by Miller and Power (2013), to the realm of influence of VaR. Through a series of semi-structured interviews with relevant actors in the field, we investigate the latent power and endurance of VaR as an accounting measure despite its apparent shortcomings and a loosening of its regulatory power. We explore its dominance within banking organisations and in financial markets (territorialising), how it is communicated (mediating), the control aspect of the device (adjudicating) and its propensity to prompt action (subjectivising). We find that the territorialising and subjectivising power of accounting measures is revealed in how VaR has become embedded in the thinking of users. This “stickiness” of an accounting technology may become an impediment to change or may affect the perception of the impending change. This case shows the latent power of accounting measures, and the reach they can acquire when developed in an under-regulated way. We find that the Miller-Power frame has resonance beyond what is considered conventional financial accounting. It highlights the need for awareness of the embedded nature of accounting measures when implementing regulatory, organisational, or market changes. Hence, we find that VaR, as a useful though inaccurate (Millo and MacKenzie 2009) risk model may have sustained longevity in its internal risk management role beyond the implementation of FRTB. We deploy a mixed methods approach whereby we use the findings from a qualitative evaluation of VaR and the market risk regulatory framework, to inform and shape a quantitative evaluation of the impact of the FRTB calculative framework on risk and portfolio management practice. One of the key findings from the qualitative study is the belief by practitioners that the FRTB regulatory framework will have limited impact on risk modelling, chiefly entailing a change of metric at the end of the risk modelling process (that is, a point estimate at 99% confidence level replaced by an average of the tail at 97.5% confidence level). This framed our quantitative analysis to examine the potential impact of the additional criteria introduced under FRTB for the authorised use of proprietary risk models (regulatory VaR/ES): P&L attribution test and desk-level backtesting. Our quantitative impact study examined the propensity for equity portfolios to pass the P&L attribution tests. We found that particular characteristics, inclusion of high market-capitalisation stocks and weightings proportional to index-weightings, increased the likelihood of passing the P&L attribution test. We also found that the FRTB desk-level backtests had low power to reject poorly performing resolution models. We argue that these results infer a change of emphasis from the ex post performance of the VaR resolution models to the ex ante role of portfolio management. This is important because it has implications for the implementation of the FRTB regulatory framework. Banks will find it challenging to meet the additional requirements of the PLA tests to secure authorisation for use of their internal models unless they recognise this change of emphasis. The difficulty in passing the PLA tests prompts two alternative actions for banks. First, that they adopt a full revaluation approach to market risk modelling. This would have significant system implications and the computation time would increase significantly. Alternatively, that they enhance alignment between the portfolio and the risk factors through the construction of the bank portfolio. These options may prove too onerous and costly and may cause banks to reconsider the efficacy of pursuing the authorisation of internal models for market risk regulatory capital. The widespread adaptation of the standardised approach is contrary to the BCBS philosophy that the use of internal models facilitate a level playing field between banks in different jurisdictions. A common characteristic in the criticism of VaR is the assumption that the same resolution model is used cohesively in each of its roles. Our qualitative study finds that practitioners’ do not perceive any conflict between using different VaR models for internal market risk management and regulatory capital calculations. A McKinsey study found that currently 50% of banks use a different model for regulatory capital purposes than for internal risk management purposes (Mehta et al. 2012). Although FRTB introduces additional desk-level backtests, our quantitative study finds that they do not incentivise the use of risk resolution models with strong forecasting ability. This is consistent with the findings of Hermsen (2010), who finds that Basel II does not incentivise the use of superior market risk forecasting resolution models. Coupling this finding with the additional calculative layers of FRTB (asset-specific liquidity horizons, weighted diversified and non-diversified ES, calibration on stress period, output floor at 75% of SA), we find that the importance of internal models in the calculation of market risk regulatory capital is diminished. Thus, FRTB will further detach the internal risk management role of VaR from the mechanism used to calculate market risk regulatory capital.