Building a Strategy Map to Drive Performance Accountability and Improvement: The Case of Audit Scotland

Introduction

Having amassed a wealth of useful data and information regarding stakeholder priorities over the years, Audit Scotland faced the not uncommon challenge of distilling this down into something that was focused and manageable. To achieve this, Audit Scotland crafted a strategy map that comprises just two core activities and nine supporting performance enablers. Each of these, which are collectively aligned to the organization‟s vision, is supported by Key Performance Questions (KPQs) and Key Performance Indicators (KPIs) - which together provide rich insights into how the organization is progressing toward its strategic goals. The strategy map is designed to help Audit Scotland make step-change improvements in performance and to provide a more streamlined and focused corporate plan. An Impact framework is also being used to assess and report the longer term benefits of Audit Scotland‟s work, which will serve to provide a more complete view of the organization‟s performance.

About Audit Scotland

Headquartered in Edinburgh, Audit Scotland was created in 2000 to provide the Auditor General and the Accounts Commission with the services that they need to check that public money is spent properly, efficiently and effectively. With about 300 full-time equivalent employees and an operating budget of £27.5 million, the organization arranges the audits of about 200 public bodies, such as the Scottish Government, 32 councils and 44 joint boards and committees (including police and fire, health boards and rescue services). Importantly, Audit Scotland is independent of the bodies that it audits.

Three types of audit

Audit Scotland conducts three types of audits: 1. An annual audit provides an independent opinion of the public bodies‟ financial arrangements and governance practices. 2. Best value audits provide an independent opinion of an organization‟s outcomes focus as well as the effectiveness of its corporate performance management and use of resources. 3. Performance audits provide an independent assessment of key areas of parliamentary or public interest, often spanning traditional sector boundaries. Performance audit reports include "Scotland Public Finances," "The role of Boards,‟ "Efficient Justice‟ and "Public Sector Pension Schemes‟. Together, these three audit types provide a detailed and systematic evaluation of the efficiency and effectiveness of how the nation‟s public organizations work and are being managed.

Identifying Stakeholder Requirements

As well as helping to improve how others work, Audit Scotland is firmly committed to continuously improving its own performance. As part of this ongoing process, during 2008 the organization conducted consultation with a broad range of stakeholders, including staff, partners and clients, in order to better understand how to meet their needs going forward. "We knew that the business and operating environment of our stakeholders was rapidly changing," says Diane McGiffen, director of corporate service. "And we wanted to ensure that we captured these changing circumstances in our corporate plan for 2009-2012." Using an external research company Audit Scotland generated substantial quantitative and qualitative information as to how its various stakeholder groups thought it was performing and what they thought its priorities should be over the following five years. "We also extended this exercise to our staff which gave excellent additional insights into how we should develop and improve," says McGiffen, whose wide ranging remit includes being in charge of business planning and performance reporting and heading up human resources and training, information technology, finance, administration and communications.

The data challenge

Although such widespread consultation provided a wealth of valuable data, Audit Scotland found itself in a position that is common across both public and commercial organizations. "The more data and information that was generated from surveys, discussions and other formats, the more challenging it became to identify and focus on the critical few things that will really make a difference to our performance," comments McGiffen. "We knew we had very good information, but we were having difficulty in distilling it into something that was focused and manageable."

A Powerful Corporate Vision

To support this distillation, Audit Scotland engaged the services of the Advanced Performance Institute (API). During the winter of 2008, API worked with Audit Scotland to create a corporate strategy map (shown on Figure 1) which identified the key strategic objectives and supporting activities that would be required to deliver to the following vision: "On behalf of the Auditor General and the Accounts Commission, we will provide assurance to the people of Scotland that their money is spent appropriately and we will help public sector organizations in Scotland to improve and perform better." This vision speaks directly to the efficiency (value for money) and effectiveness (performance improvement) strands of Audit Scotland‟s responsibilities. As with any useful vision, this became the anchor of the subsequent internal discussions that focused on crafting the strategy map. However, it should be noted that in the process of creating the strategy map the vision was slightly amended and updated. "This was a useful outcome of the workshops conducted to create the map," says McGiffen. "The vision we have now captures the essence of the priorities that our stakeholders and clients have identified for the next five years."

Stakeholder priorities

The priorities identified by the stakeholders groups, which have therefore become the key priorities of Audit Scotland, are: • Deliver more streamlined audits in partnership with other scrutiny bodies. • Maximize our contribution to the improvement of public services. • Increase the impact of our work. • Become a centre of excellence for public audit. • Improve the transparency of our costs and governance arrangements. The priorities are based on the following issues that stakeholders are grappling with, that emerged through the consultation process: • increasing pressure on public spending • Increasing emphasis on more efficient working

• The continuing need for high standards of corporate governance, financial management and control • The need to deliver more efficient and effective scrutiny and accountability.

Creating the Corporate Strategy Map

The first step in creating the strategy map that would deliver to the vision and stakeholder priorities involved API conducting focused and individual interviews with each member of the executive management team and the assistant directors as well as the organization‟s chair John Baillie and another board member. A core benefit of one-to-one interviews is that it enabled the external facilitator to gain a clear picture as to how individual senior managers viewed the drivers of organizational strategic success. Any differences were debated within the workshops conducted to create the strategy map, with the ultimate goal of reaching consensual agreement as to the critical few drivers of success.

Strategy mapping workshops

Audit Scotland‟s management group created a draft strategy map in two dedicated workshops. A key role of the external facilitator was to ensure the selection of truly strategic objectives, rather than goals that are essentially operational. A further role was to facilitate the creation of cross-organizational objectives instead of ones that could be siloed - that are of relevance and interest to just one part of the organization. Audit Scotland‟s management team wanted to ensure that the strategy map would galvanize the whole of the organization in unison toward step change performance improvement.

In-house consultation

Wide-ranging staff engagement was seen as crucial for securing buy-in to the strategy map concept. "Through focus groups and discussions with small groups of staff we gained feedback on what they thought of the map, what they thought were its strengths and weaknesses and where they saw areas for improvement," recalls McGiffen. "This showed us how closely the management team‟s views were aligned with those of their colleagues and the staff feedback was used to inform the final map." According to McGiffen, the clarity and simplicity of the map helped most staff to understand and talk about what it means to them "Within the group discussions it became evident that colleagues could see how it all fits together, what Audit Scotland is trying to achieve, how it will be achieved and what they must do in their own jobs to make it happen." This, she adds, was largely due to the map‟s simplicity and accessibility.

Ensuring the map is simple and focused

As an illustration of simplicity, the map comprises just eleven elements: two core activities (which Audit Scotland calls objectives) and nine performance enablers (which Audit Scotland calls supporting activities). A prime driver of formulating the strategy map was to distil and make manageable a mass of data and information, Audit Scotland rightly realized that there would be little benefit in creating an elaborate and complex map. Indeed the experience of other companies shows that complicated maps do not provide sufficient focus onto the key drivers of success and rarely survive over the longer term due to being too unwieldy and unmanageable - a fate Audit Scotland was determined to avoid. Although simple, the map is far from simplistic. Rather, each objective and supporting activity serves as a doorway to a host of performance management and measurement focal points - primarily captured through Key Performance Questions (KPQs) and Key Performance Indicators (KPIs) - that work together to provide an overall picture of how well Audit Scotland is performing and, ultimately, to steer the organization toward the delivery of its vision and priorities (For more information on KPQs and KPIs see the API white paper "What are KPQs‟ and the API management overview "Key Performance Indicators‟). Although KPIs are commonly used within organizations, KPQs are substantially less so. So as an explanation, a KPQ focuses on and highlights what the organization needs to know in terms of executing existing strategic objectives. KPQs enable a full and focused discussion on how well the organization is delivering to these objectives and serve as an important bridge between organizational goals and KPIs. Indeed KPQs are used to provide a performance context to KPIs and help to more effectively prioritize the indicators chosen, and this was certainly true of Audit Scotland. "People in most organizations, and we are no exception, have quite rightly developed strong disciplines in KPIs as a way to monitor and improve performance," says McGiffen. "KPQs do not replace KPIs but are an enhancement. KPQs help us to capture a much richer set of information than cannot be gained solely from performance data." Today, KPQs are used to discuss performance at many organizational levels within Audit Scotland and Audit Scotland is refining and developing its collection and reporting of data to address the KPQs. A key initiative was a Performance Management Festival held over two days to bring focus and attention from across the business to the performance reports. We will now explore how the objectives/activities on the map, KPQs and KPIs work together with Audit Scotland‟s strategy management framework.

Two strategic objectives

As cited, Audit Scotland‟s strategy map includes just two core activities (called objectives in their case), these being: "we will conduct excellent risk based audits of the public sector and report them in public," and "we will systematically identify and promote good practice to help public bodies to improve." Although both objectives are equally important, a key message that emerged from the extensive stakeholder consultation was that they were looking to Audit Scotland to do more to help them to improve performance. The KPQs to support the "improve‟ objective (and note that KPQs and indeed KPIs are still work in progress within Audit Scotland, while they test which are the most powerful questions and indicators to focus on) presently include: "to what extent has our work led to improvement‟ and "to what extent are we identifying and actively promoting good practice (or better ways of doing things) to help public bodies improve. Note that the KPQs focus on both "lagging‟ (that is how we have performed) and "leading‟ (how we will improve performance) dimensions, enabling a fuller overview of past, present and likely future performance. KPIs for these KPQs include "impact reports‟ (see later in this case study for details of Audit Scotland‟s impact framework); "client feedback‟; "number of presentation at meetings, events and conferences which include best practice guidance,‟ and "performance audit reports with good practice recommendations‟ (see Figure 2 for examples).

Supporting Activities

How Audit Scotland achieves its objectives is largely through the delivery of enabling objectives (called supporting activities in their case). In creating the map, the senior team strived to ensure that there was a balance of focus on what matters to stakeholders, what matters to staff and improving some of the key business support processes to support that work.

Stakeholder-facing performance improvement

For a stakeholder-facing improvement, consider the activity: "We will closely engage and communicate with our key stakeholders, clients and partners, and other scrutiny bodies.‟ Audit Scotland recognizes that it is only through close engagement and communication that it can effectively identify appropriate topics for its audit work. Closely engaging and partnering with other scrutiny bodies and improvement agencies also helps to avoid duplication of effort and maximizes Audit Scotland‟s contribution to the improvement of public services. To support the improvement objective it is also recognized that it must clearly communicate its findings and recommendations to ensure that its messages are heard and its recommendations are followed up. KPQs used for this activity are "to what extent are we keeping our stakeholders informed about our programmes of work and their role/implications‟ and "how well do we engage the people we work with.‟ KPIs include "positive stakeholder feedback‟ (engagement levels, clarity about and acceptance of roles), "website statistics (downloads, click throughs) and the "number of shared risk assessment frameworks in place.‟

Internal process improvement

Step-change internal business process improvements are largely captured by the supporting activity: "We will deliver our work and manage our resources efficiently and effectively and will be clear where we need to improve. We will have excellent governance procedures and will monitor and report on the impact of our work.‟ This activity sets out to ensure that Audit Scotland has strong internal governance processes in place and that it is deploying best practice public management principles and procedures. To this end, Audit Scotland carries out a programme of internal best value reviews to improve its efficiency and effectiveness and manage its risks. The KPQs for this activity include "to what extent have we effectively and efficiently managed our resources to deliver our outputs‟ and "how effective are our internal governance arrangements for delivering our objectives.‟ KPIs include: "percentage of audit reports delivered on-time and to budget,‟ "reducing our carbon footprint by one per cent,‟ "percentage of invoices paid within 30 days and "internal best value reports published.‟

Staff performance improvement

Moving onto improving staff performance this is captured by several supporting activities, including "We will maintain and develop our professional skills and competencies‟. As with any high-performing organization, Audit

Scotland recognizes that it can only deliver high-quality work if its staff have the right skills and competencies to do their job well. To achieve this, the organization strives to embrace an ethos of continuous improvement and continues to invest in the development its people. The organization has recently revised its competency framework and has embedded this into its Performance Development System. Regular individual performance reviews are used to assess and follow-up on training and development needs. The KPQs for this activity include "to what extent are our employees engaged and motivated‟ and "to what extent do we have strong and effective leadership in the organization‟. KPIs include "staff survey results,‟ "staff focus group feedback‟ and the development of "360 degree feedback (through which employees are appraised by a range of people, including their managers, colleagues and direct reports)‟.

Knowledge management

A particularly interesting supporting activity, in that it clearly demonstrates how Audit Scotland is working to improve its own internal capabilities in order to help its stakeholders improve is "We will gather, organize and share knowledge and intelligence.‟ The organization recognizes that the basis of its work is having the right knowledge and intelligence. Therefore it is working diligently to ensure that the right information is systematically gathered, that it is organized in the most appropriate way, and shared among staff and externally with stakeholders. KPQs for this activity are "to what extent do we hold the information that we need‟ ' to what extent are we sharing our key findings internally‟, and "to what extent are we able to find and share the information that we find‟. KPIs include "focus group feedback‟, "intelligence reports‟, "publication of technical bulletins‟, "number of briefing reports produced and shared internally‟.

Creating Strategy Maps for Specific Activities

Audit Scotland is presently reviewing and developing how it gathers, organizes and shares knowledge and intelligence. This is leading to some interesting applications of the strategy map concept, as McGiffen explains. "Our strategy map identifies the key support activities where we need to focus attention and commit resources," she says. "What we are doing now is creating strategy maps for specific areas where we want to prioritize major performance improvements, such as information and knowledge management and shared learning." She continues: "Our stakeholders are increasingly looking to Audit Scotland to share knowledge and information quickly from the full range of our work and advanced knowledge sharing and collaborative capabilities. However, our technologies and methodologies don‟t enable us to do what we need to do to meet stakeholder requirements. We have started to invest in this area and a strategy map should prove a powerful mechanism for prioritizing our investments against identified stakeholder needs." Environmental performance is another area for which a dedicated strategy map will be created.

Performance Reporting

With the strategy map now the core tool for driving performance improvement, Audit Scotland ensures that it is closely managed on a day-to-day basis, and this is the responsibility of McGiffen and a small team from corporate reporting that is skilled in data analysis. The team reports performance to the management team and board on a quarterly basis, who particularly look to see how objectives and activities are improving and to focus on areas that are proving challenging and where corrective action might be required. From a reporting standpoint, the strategy map and related KPIs have also enabled McGiffen and her team to cut the corporate plan down from a lengthy document to something concise and focused and that is more accessible to external stakeholders and internal staff respectively. Performance Management Festival One innovative approach to engage people in performance reporting was to stage a "Performance Management Festival‟. In late 2009 Audit Scotland invited key staff for a two-day festival of performance. The aim was to bring together KPI owners, performance analysts, and directors and board members to agree performance reporting formats. Using an off-site location with music, guest speakers and performances this innovative event, facilitated by the API, was designed to make the topic of measurement and performance reporting interesting, engaging and fun. During the two-day event staff re-designed performance reports using the latest techniques of information communication and data visualization. Within these reports, headlines and narratives (essentially a performance story) were used to capture the key messages, using ideas and graphic/textual formatting from magazines and newspapers. These new performance reports were then "trialed‟ and improved during a staged mock performance review meeting with the directors.

Impact Framework

Performance improvement is also being galvanized through the deployment of an impact framework to assess and report on the longer term impact of Audit Scotland‟s work - in particular its performance audits - and to capture evidence of that impact. The organization has identified four areas where it expects its work to have an impact and contribute to improvements: • Assurance and accountability • Planning and management • Economy and efficiency • Effectiveness and quality. The impact framework describes the types of measurable change expected to result from Audit Scotland‟s work within the different categories of impact and includes examples of how individual reports may contribute to these changes. The impact framework is expected to dovetail well with the strategy map in providing a complete view of how Audit Scotland is performing now and is likely to perform in the future.

Conclusion

In conclusion, McGiffen summarizes the role that the strategy map has played in improving the performance of Audit Scotland. "From the beginning we knew that we wanted our Corporate Plan to be simpler and more relevant and meaningful for colleagues." she says, "From an external perspective we also knew that as an organization we wanted a framework that would help our stakeholders understand what we do and what we have to be best at." "The strategy map is helping us to meet our external and internal goals," she continues. "But in the final analysis, it is perhaps most important that the map has better positioned Audit Scotland to be able to demonstrate to stakeholders and clients that we are doing what we promised that we would do."

Bernard Marr is a globally regognized big data and analytics expert. He is a best-selling business author, keynote speaker and consultant in strategy, performance management, analytics, KPIs and big data. He helps companies to better manage, measure, report and analyse performance.
Key Performance Indicators: The 75+ Measures Every Manager Needs to Know
Bernard Marr
Performance indicators are essential tools which will tell you if your business is on target or veering off course. Using the right indicators will help you deliver the right results. Key Performance Indicators cuts straight to the 75 + KPIs that ...
KPI Library
Discover the right Key Performance Indicators for your business.
Videos
The Advanced Performance Institute