Performance measurement in the Government of Alberta

Despite the proposed positive aspects of performance measurement, there have been numerous concerns raised about the limitations of being able to measure in a public sector environment. While some people tend to raise more technical concerns, others raised more philosophical concerns about the legitimacy and authenticity of the performance measurement process, given the measures are publicly reported in the government’s business plans and annual reports. In this sense, the legitimacy of performance measurement is threatened because the measures, targets, and results are perceived to be “massaged and manipulated” by management, a central agency, or a communications department. In other words, high-risk measures, such as those that fluctuate, are difficult to attribute, never meet their target, and have a low citizen satisfaction rating, are unlikely to get or remain in a business plan. The third challenge to measuring performance in a government setting is that the external performance measures and targets are linked to department, deputy minister and individual performance plans. This final challenge threatens the validity of the performance measurement framework in the sense that civil servants are likely to choose performance measures and targets that are easy to measure, are stable, and the targets are met or surpassed each fiscal year. It is this subjectivity and the technical challenges of performance measurement that lead to the questioning of the legitimacy and authenticity of reporting on performance in a public sector setting. This subjectivity of both performance and results contributes to the paradox of public reporting. On the one hand, a government can be praised for being transparent in its plans; on the other hand, it can be criticized for publishing politically safe and strategic information for fear of retaliation from the media, opposition parties, and disgruntled citizens. It is this paradox that will be explored in the article under the realm of bureaucratic propaganda.


Introduction
There are a variety of ways to define performance measurement.At a basic level, performance measurement is a qualitative or quantitative measure designed to assess performance against a goal.Governments have always used some type of measurement to assess their programs, services, and overall popularity, 1 but in the last decade, measuring performance has taken on a more formal and public approach than in previous times.Evolving from the fields of auditing, budgeting, and policy/program evaluation, performance measurement was developed in many government jurisdictions to support and complement management and policy decision-making in the late 1980s and 1990s. 2 In the past several decades, numerous monographs and articles have been written about performance measurement, 3 and most provide general advice on how to develop input, output, and outcome measures; how to develop benchmarks and baselines; what resources are needed to develop a performance measurement framework; and what some of the challenges are to developing, implementing, and evaluating performance measures in the public sector.Furthermore, performance measurement has been written from the perspective of all levels of government (local, regional, provincial, and federal) and numerous authors have also addressed a wide variety of policy areas in their research.
Alberta was the first provincial government in Canada to adopt a publicly reported, results-based performance measurement framework.Performance measurement was an integral component of the newly established business planning process, and in a broader context, it was also part of the accountability framework that was designed to improve accountability between civil servants and elected officials and between government and citizens. 4Specific to the relationship between government and citizens, there seemed to be the assumption in the early years of the Klein government that in order to be "good citizens," citizens needed information from the government to be fully knowledgeable about the actions and goals of government.In this sense, the business plan, the document outlining government-wide and departmental performance measures, was viewed by the Government of Alberta to be an accountability contract between the provincial government and Albertans.
Performance measures, whether in the Government of Alberta's annual reports, Measuring Up reports, or in the business plans, were intended to provide citizens with information about the government's performance.According to the Government of Alberta, the positive impact of reporting publicly on their performance is that it would enhance transparency of government actions and decisions, strengthen accountability between the state and citizens, improve strategic planning so plans are developed with long-term citizen objectives in mind, monitor external contracts to improve accountability and to ensure fairness, promote interdepartmental communication and collaboration, and provide motivation for civil servants to improve services and programs for citizens. 5In this sense, the business plans and the performance measures, as reported by the Government of Alberta, were to be the messengers of democracy.The business plans and performance measures were developed to relate complex information to citizens in a timely and easy-to-read manner, at the same time as acting as an accountability contract between the state and citizens.Central to the development of government business plans, a monitoring system, and an evaluation report (the Annual Report), was the assumption that if government did not meet the expectations of its citizens, they were rightly entitled to not elect them as the government in the next election.
Despite the proposed positive aspects of performance measurement, there have been numerous concerns raised about the limitations of being able to measure in a public sector environment.While some people tend to raise more technical concerns, others raised more philosophical concerns about the legitimacy and authenticity of the performance measurement process given the measures are publicly reported in the government's business plans and annual reports.In this sense, the legitimacy of performance measurement is threatened because the measures, targets, and results are perceived to be "massaged and manipulated" by management, a central agency, or a communications department.In other words, high-risk measures, such as those that fluctuate, are difficult to attribute, never meet their target, and have a low citizen satisfaction rating, are unlikely to get or remain in a business plan.The third challenge to measuring performance in a government setting is that the external performance measures and targets are linked to department, deputy minister and individual performance plans.This final challenge threatens the validity of the performance measurement framework in the sense that civil servants are likely to choose v e ndre di , l e 4 m a rs 2016 performance measures and targets that are easy to measure, are stable, and the targets are met or surpassed each fiscal year.It is this subjectivity and the technical challenges of performance measurement that lead to the questioning of the legitimacy and authenticity of reporting on performance in a public sector setting.
Before assessing the interview results, it is important to further explore the highly political nature of defining or assessing performance in a government setting.For example, Arie Halachmi and Geert Bouckaert argue that depending on what political suasion or ideology one supports, performance becomes a highly subjective term.The authors argue that because of the different ideologies that public choice theorists, classical Marxists, neo-Marxists, principal-agent theorists, welfare economists, and other theorists hold about the role of government, "performance must be viewed as a set of information about achievements of varying significance to different stakeholders." 6So while there may be a general definition of performance measurement, moving beyond the basic understanding becomes highly political in the sense of what is believed it can accomplish and what is the ultimate purpose.
Related, and discussed to a certain extent earlier, the authenticity of performance measurement is questioned because of the subjectivity of performance itself.In his recent paper on performance measurement, Paul Thomas comments on how challenging it can be to define performance because of the subjectivity of the term. 7Indeed, the process of measuring performance is a highly subjective task depending on the stated expectations, the established targets and goals, external variables, the quality of leadership, and a variety of other reasons depending on the assessor's values and biases.The subjectivity of performance is inherent to a political environment, which makes performance reporting a risky endeavor for any government.For example, reporting to the public in a political environment is precarious given the traditional role of the Opposition to critique the party in power and the media to report on issues that will attract customers which tend to be those that expose the government's wrongdoings. 8here may also be interest groups willing to protest if the government is not performing according to their own expectations and citizens in general may express their own displeasure and distrust in an upcoming election.
Similar to the previous arguments made about the subjectivity of performance, Barbara Wake Carroll also argues there are different ways for a government to define a result 9 and that it is "infinitely more difficult in the public sector [than in the private sector] because of the nature of public good." 10The author also comments that the decisions of which result to use and "who decides whether the results are good or bad…are intended to affect, the way things are done and evaluated." 11In the end, Carroll argues that the decision on what definition to use and the assessment of the result is a political decision, not a technical one.This subjectivity of both performance and results contributes to the paradox of public reporting.On the one hand, a government can be praised for being transparent in its plans; on the other hand, it can be criticized for publishing politically safe and strategic information for fear of retaliation from the media, opposition parties, and disgruntled citizens.It is this paradox that will be explored in the article under the realm of bureaucratic propaganda.
In a general sense, Harold Laswell wrote that propaganda is the making of deliberately one-sided statements to a mass audience. 12David Altheide and John Johnson further distinguish propaganda from bureaucratic propaganda and state that the definition of the latter is "any report produced by an organization for evaluation and other practical purposes that is targeted for individuals, committees, or publics who are unaware of its promotive character and the editing processes that shaped the report." 13Central to bureaucratic propaganda is the tactic to use truth for strategic purposes.In other words, how truth is regarded depends on what a government wants the truth to do.For example, truth can be manipulated to obtain funding, legitimacy, and communicate performance regarding what is being done and what is being accomplished.The two authors further state that bureaucratic propaganda can either be false or true but what matters most is the appearance of legitimacy. 14Moreover, they argue that legitimacy is developed and communicated by a government presenting reports as though they were done scientifically and therefore depict objective truth.Within this context, according to the two authors, there is no place for failures, waste, and inefficiency, or in general, any data that could be construed as contrary to the organization's stated goals. 15This method of reporting ties in with new public management as it attempts to take the "politics" out of planning and reporting and it also complements the Klein government's mantra, to run government like a business.
Ergo, the question of whether or not performance measurement is truth or propaganda arises and the answer, according to the interview results, depends on the position of the respondent.Before discussing the research results though, it is important to give some background to the methodology used to support the findings and to briefly introduce the reader to the Government of Alberta performance measurement framework.

Methodology
In this article, an analysis of publicly reported performance measures will take place in the context of interviews conducted with retired and current civil servants, politicians in the Government of Alberta, members of the business community, and the media in Alberta who are familiar with the business planning and performance measurement.The interviews covered a wide range of material, focusing on general themes related to accountability and business planning.The intent of the interview process was to ask what aspects of business planning have worked and what aspects have been unsuccessful with a view to determining the impact of business planning on accountability.Other secondary themes that arose from the interview process included the transition from traditional methods of planning to the new business planning model, reasons for the implementation of the business plans, the development and implementation of business plans and performance measures, the strengths and challenges of measuring in a public sector environment, and how this method of planning affected the relationships between civil servants and elected officials and between the government and citizens.
Overall, 76 people were interviewed and of these 70 participants, four participants were interviewed twice given their expertise and experience in the business planning area and most of all, their willingness to be interviewed for longer periods of time than other interview participants.Each interview was conducted in person and the interviewee was informed that the results of the interview would respect the confidentiality of the respondent.To narrow the scope of the study, nine ministries were selected and these ministries were chosen to reflect the diversity and complexity of services, the variance of budgets (big budget versus small budget), the physical size, and the structures in place to support the business planning process in the Government of Alberta.
The interviews differed according to each individual's experience; however, a set of questions was asked of each participant in order to provide a comparative analysis.There were two example question formats developed depending on the length of time that was available for the interview and the position of the interviewee.For example, a person from the private sector who was a representative on the Financial Review Commission would have a different experience and knowledge base than a person who works in the Alberta Public Service in a planning position.Further, an elected representative of the Progressive Conservative Party would have a different perspective on accountability compared to a member of the Liberal or New Democratic Party.To address the time factor, a list of core questions was developed to ensure that for each interview participant, at least the most important questions would be asked.
One of the areas of assessment, the Auditor General's reports, will not be analyzed given space constraints; however, this aspect of public reporting and monitoring is addressed (as it relates to the Government of Alberta) within my doctoral dissertation and numerous other authors have also addressed the changing role of the Auditor General in Canada and in other jurisdictions with respect to planning, performance measurement, and ongoing monitoring. 16e performance measurement framework in the Government of Alberta The purpose of performance measurement in the Government of Alberta is multi-faceted.Initially, the primary purpose of performance measurement was to improve accountability as the measures were a fundamental component of the accountability framework.For example, in an early paper on performance measurement in the Government of Alberta, Rich Goodkey and Ken Ogata, two civil servants who were employed in Alberta Treasury in the early to mid 1990s, argue that "performance measurement was just one of several reforms initiated to make government more open and accountable to the public." 17In the later 1990s, politicians and Alberta Finance began to publicly state that another purpose was to improve overall management and planning in government.While this latter purpose was still visible in early performance measurement documents, this message became increasingly important as the Government of Alberta adopted a results-based approach to measurement and planning in general.
Principles, as they apply to performance measurement in the Government of Alberta, are guidelines for the establishment of performance measures for the individual ministry business plans.The guiding principles, as outlined in Alberta Finance's Measuring Performance: A Reference Guide, are the following: …focus on results -determine the effects programs are having rather than measuring what has been produced; a few key measures per ministry -provide a snapshot of the ministry's performance for its core businesses; measures developed by ministry program officials -programs officials know their business the best; measures owned by ministries -ministries held accountable for the measures they develop and the results they achieve; measures should be free from bias -report both good and bad performance; and, work with the Auditor General -to ensure the selection of valid and objective measures. 18cently, at a conference held in November 2003, Julian Nowicki, Deputy Minister of Executive Council with the Government of Alberta, stated the four principles of performance measurement: specify desired measurement results for each goal, measure progress towards desired results, report the results, and use the results to improve effectiveness. 19It was not stated whether these principles replace or supplement the earlier principles.
As noted earlier, business plans have also become a way to communicate to the general citizenry about the government's performance at the department and government-wide level.In a public effort to become more accountable and transparent, the reporting of performance measures is considered by the Government of Alberta to be an educated way for citizens to judge a government's annual performance.For example, Alberta Finance states that a "government that reports its own performance to citizens, rather than totally relinquishing that task to the media, has far more control over the manner in which information is disclosed and greater opportunity to describe its response to particular problems." 20Instead of the smoke and mirrors typically associated with politics, a public performance measurement framework can allow a government to establish a management, communications, and accountability relationship with its citizenry.

Legislation and performance reporting requirements
Prior to 1993, a formal performance measurement framework did not exist within the Alberta government.The interview results found that while there were some departments that used performance measures to assist in program management and performance monitoring prior to the establishment of business plans in 1993, it was not a government-wide practice nor were the measures made available to the public. 21Reporting on the government's performance relied on traditional government tools such as annual reports and government speeches.
In 1993, when Klein's Conservatives won the provincial election, numerous changes were made to the structures and processes within the bureaucracy.One of the most significant changes was the development of an accountability framework of which the two main components were business planning and measuring performance.Committed to improving accountability, Alberta legislated performance measurement in conjunction with the development of the business plan. 22The Government Accountability Act was passed in 1995 and this legislation established requirements for the Alberta government and individual ministries to report on their performance to the Legislative Assembly and to the public in business plans and annual reports.Business plans and performance measures are required by legislation as set out in the Government Accountability Act (7.3) and the Act states that the government business plan is to include the following: (a) the mission, core businesses and goals of the Government; (b) the measures to be used in assessing the performance of the Government in achieving its goals; (c) the performance targets set by the Government for each of its goals; (d) links to the ministry business plans. 23t only does the Government of Alberta have to develop performance measures, they also have to develop targets and publicly report on an annual basis.For example, in the consolidated annual reports, there has to be "a comparison of the actual performance results to the targets included in the government business plan ...and an explanation of any significant variances." 24 provide a comprehensive picture of performance measurement to citizens, the Government of Alberta developed an annual report titled Measuring Up. 25 Released with the annual report since 1994, the primary focus of Measuring Up is to communicate to the public on the performance of the Government of Alberta.The report contains information on the government's performance and includes both financial and non-financial measures.Within the report, the Government's current core measures and results are compared to the previous year's results to provide context but there is no further analysis.As noted on the Government's website, "there are no judgments in Measuring Up about whether the results are good, bad or indifferent.We leave Albertans to make up their own minds." 26Ergo, if a department does not meet their target or if a department decides to eliminate and replace a performance measure, there are no explanations as to why these changes take place in Measuring Up as only the measures are presented.In recent years, there have been several initiatives to provide more information to assist with Albertans' assessment of the Government of Alberta's performance.For example, in the 2003 Measuring Up report, the Government of Alberta notes that, Supplemental measures are also provided to give Albertans more information on progress made towards achieving the goals.Explanations of how major influences or external factors affected performance results are also included.This helps readers better assess performance over the past year.One example is the land quality core measure, which shows a need for improvement.The biggest factor affecting this measure was the drought in southern Alberta. 27 addition, detailed information about departmental measures can be found in each of the annual reports and includes such information as other performance measures, the methodology behind each measure, and a comparison of results with the preceding year(s).

Structure of serformance measurement in the Government of Alberta
The structure of performance measurement can be discussed in numerous ways.Regarding the vertical structure of government, the Government of Alberta has developed an integrated approach to performance measurement where government-wide goals and strategies exist and cascade down into the departmental business plans, then to a department's operational plans, and finally, to the individual performance plans.In addition to the vertical method of measurement, the Government of Alberta has also developed cross-government goals, measures, and targets.According to the 2002-03 Government of Alberta Annual Report, the cross-government initiatives are the Aboriginal Policy Initiative, the Alberta Children and Youth Initiative, the Economic Development Strategy, and the Health Sustainability Initiative.As noted in the Annual Report, information on the government's progress in relation to the objectives and targets established for each of the four initiatives is reported on a yearly basis. 28Another way to describe the performance measurement structure in the Government of Alberta is to describe the different levels.Barbara Wake Carroll and David Dewar note that performance management can be difficult to understand because it typically addresses three types of assessment: One is the evaluation of programs or policy at the broad governmental or political level and includes a political consideration of basic objectives.The second is the implementation and management of an individual policy or program.The third is the assessment of the performance of individual employees. 29e Government of Alberta eventually adopted these three levels of assessment but in 1993, they developed their own "made in Alberta model" to complement their own performance goals and strategies.According to Alberta Finance, when performance measures were first developed, the three tiers of measurement consisted of core government measures, key ministry measures, and internal management measures.At the outset of the performance measurement framework in 1993-94, there was no formal model for assessing a civil servant's performance and measures.Individual policies and programs were not given significant attention, as these were believed to be intertwined into ministry business plans and internal management measures.
The core government measures represent the Government of Alberta's macro level measures and these measures report on high-level outcomes that are deemed to be the priorities of Albertans.Within the Government of Alberta's business plan, the three core businesses, or priorities of Albertans, are people, prosperity, and preservation.Each of these core businesses has goals and strategies and corresponding performance measures that originate from the related departments.This first stage is also known as the level at which Albertans can assess the results of the government's overall performance.The second tier consists of key ministry measures.Alberta Finance notes "each ministry has selected several key measures to provide Albertans with an overview of the ministry's performance for its core businesses.These measures focus on the outputs and outcomes of ministry policies and programs, and provide background information which feeds into the core government measures." 30Finally, the last original tier is the development and implementation of internal management measures that have recently become part of departmental operational plans.These types of measures assess the performance of internal programs and services and are typically developed to provide supporting data for higher level measures such as the core or ministry measures. 31l of these tiers still exist in the Government of Alberta, but other layers of measurement now exist as well.For example, performance measures to assess the performance of external contracts and individual employees have been developed to provide a more complete assessment of performance in the Government of Alberta.Recently, the Government of Alberta also changed its public planning structure and they now produce a strategic plan that is available on their website. 32The strategic plan projects what the Government of Alberta, with input from Albertans, want in the year 2025.As outlined in the plan, the four pillars of the plan are unleashing innovation, leading in learning, competing in a global marketplace, and making Alberta the best place to live, work and visit. 33The strategic plan links to the rest of the planning process and the vision of the strategic plan is to act as a guide for the rest of the plans (business, operational, and individual).Alberta Finance states that this level of integration "ensures that all of government is 'pulling together' and that all strategies are linked to Alberta's vision and 20-year strategic plan.While important as a guide, the strategic plan does not have any additional performance measures; instead the task of performance measurement and reporting remains within the 3-year business plans, annual reports, operational plans, and individual performance plans." 34rformance measurement in the Government of Alberta: technical and political limitations on providing the truth In this section of the article, the results of interviews conducted will be discussed as they relate to performance measurement and reporting.The purpose of this section is to demonstrate the technical and political challenges of performance measurement faced by the Government of Alberta.
Technical challenges: the truth given the circumstances One of the most popular themes that arose during the interview process was the challenge of developing a performance measurement framework from ex nihilo.Where other jurisdictions conducted extensive research before implementing their performance measurement systems, 35 Alberta developed its framework in a relatively short time with little background research done.Several of the interview respondents stated that one of the lessons learned from Sir Roger Douglas' visit to Alberta in 1993 and his book, 36 that was distributed to all MLA's in the early months of power, was that it was important to implement change quickly rather than in an incremental manner.Ergo, while the implementation of the business planning process and performance measures was relatively quick given the magnitude of change required at the structural, procedural, and cultural level, the cost was that there were still gaps in knowledge about performance measurement such as determining what the measures, targets, and baseline numbers should be for their specific department.
When performance measures were first being introduced into the public sector, there was little expertise and experience about the development, monitoring, and reporting of performance measures.One of the biggest challenges was determining what measures to use and finding the data to support a measure since this type of information may not have been kept or researched in the past.For example, one of the senior public servants commented on the need to find data to support performance measurement in his department: We were one of the first governments to do this.There is a lot that we still need to do and a lot we need to develop.We have to get better at our measures and we have to get better at our data.Our data in some cases is not good as we are using a lot of proxies.We have never seen a need to collect that data because there has never been a purpose in collecting it…Who cared about the population in your region, until suddenly you got funded on your populationnow people pay attention to population...As we have gotten better with measures, expectations, and accountability, we have gotten better with some of those other skills, but we are still learning. 37There were also a few instances where performance measures were dropped for a variety of reasons (e.g.too unstable, not measuring what is important, and changing mandates), and respective departments were then, in some cases, required to develop a new measure.Again, if information was not being collected to support a measure, then the measure's target may be deemed to be "under development" in the departmental business plan or according to several interview respondents; the measure remained internal until proper methodology was created and implemented.Furthermore, a few civil servants commented on the lack of available data from Statistics Canada to support the establishment of baselines and ongoing measures; a few civil servants also commented on the timeliness, and hence relevancy, of this data.Finally, all of the interview participants who were able to respond to the performance measurement questions in this study commented on the difficulty of choosing which measures to include in the business plan.One civil servant notes, What are the right things to measure?If somebody could somehow wave the magic wand and tell a department, branch, or division in government or the government as a whole, these are the ten things you should be measuring… that this is the right thing to measure, you could probably make a million dollars.I m still not sure that we are measuring the right things but we are getting there.Anyone who thinks this can be done overnight is living in a dream world, because we have been at this for 4-5 years and we are just now getting to the point where we are measuring the right things. 38go, one of the technical challenges was developing performance measures, targets, and baselines from scratch in areas that were traditionally not measured and finding data to support the measures that was timely and sound.
Related to the quest for valid and timely data was the fear of duplication and "tiring out the survey respondent."Several of the interview respondents made comments about how citizens complained about getting too many surveys about government services and programs in the early 1990s.Since all of the departments were required to develop performance measures at the same time, one of the most popular ways of getting information was to survey direct clients of services and programs provided by a department.Civil servants were concerned that because there was a widespread need to gather information for all of the departments to support performance measurement, citizens may experience "survey burnout" and not want to participate in surveys, even if it meant greater transparency and improved accountability in the long run.Further, interview respondents also stated that surveying was a new methodology for many departments and again, this was another skill, if not contracted out, for civil servants to learn in the midst of downsizing and restructuring.Other problems such as only being able to survey clients they have information on and the inability to survey citizens who do not use the services were other problems they faced.
An additional challenge for civil servants was determining what type of measures they thought the public would be able to understand and care about.For example, one civil servant notes that, "It is hard to pick one that we can even measure and that is easy to measure.The other thing is it should mean something to the public and it should be something they can understand." 39One of the key persons behind the development of the performance measurement framework stated that it was important to develop and report on measures that the public really wants to know about.He further stated his test for a performance measure that was to be included in a business plan was to question if, "the stakeholder groups, legislators, and media find these things useful for public understanding and are the performance measures part of the grand scheme of things to make the world better." 40Interestingly, most of the civil servants, politicians, members of the business community, and the media who were interviewed also questioned whether or not citizens even knew about the business plans and if they did, further questioned if they had the level of comprehension to understand what the government was doing.Related, a further concern raised by the opposition parties questioned the lack of citizen involvement in the business planning process. 41In this sense, the business planning process was not viewed to be a participative process but a passive one where the citizens were to receive the information from government but have no or little opportunity to shape the original information contained in the plans.
One of the most challenging aspects of performance measurement in a public sector environment is attribution.Attribution can be described as to what extent a program or service contributes to impacting a performance measure.For example, an issue that was frequently discussed amongst the interview respondents when talking about attribution was teenage pregnancy.One of the civil servants stated it was essential that this be considered an important societal indicator for the Government of Alberta.At the same time, it was also important that each level of government, and possibly other organizations, identify their level of responsibility towards that indicator.Further, it was argued that one specific government should not be politically punished if an indicator of this nature produces negative results; instead, a dialogue should exist between the responsible organizations to determine the change in results and to focus on solutions, and not finger-pointing.One of the government members expressed the need to work with other levels of government, especially the federal level, to discuss the responsibilities of each level of government regarding performance measures that cross jurisdictional boundaries. 42other example civil servants would often discuss as it relates to attribution is the crime rate.Despite the concern about what jurisdiction should be accountable for what, the interview participants also recognized the need to measure and report on these "boundary-challenged" measures.One of the civil servants argued that, ...in reporting, you have to get over the hurdle of reporting things that you only try to influence as opposed to control.I think, generally speaking, the government is on side.So, if you are looking at what is an indicator of success in crime rate -is it a decrease in crime rate?What is our contribution?How much do we contribute to our programs so that they decrease?It is a very difficult question. 43rther challenges to attribution and performance measurement are the breadth and scope of the chosen performance measures.For example, one of the senior civil servants notes that "If your indicators are so broad and so global that you have no idea what, if any influence, you have on them, then it may be interesting information but it is not performance measurement." 44On the other hand, the same civil servant argues if government sticks with what it can directly control, then the government may not be measuring the right thing.The interview participant responded to this conundrum by stating, "It's a matter of balance." 45Unfortunately, some of the broad societal indicators a government may use may not fit well with the current electoral system.For example, if a government is required to have an election within five years of its mandate, some indicators, such as family violence, impaired driving, and health will take a longer time to realize the impact of government policy and programs on the performance measurement system.One of the civil servants stated at the beginning of the Klein mandate, "Fiscally, it was easy to have a quick outcome, but measuring the social impact of government is going to take some time." 46lated to determining what performance measures to report on is the ability to change performance measures if they are deemed to immeasurable and if a better measure is apparent.When the interviews were conducted, the business plans and performance measures had been in place for 5-6 years.During this time, interview participants stated that there had been some criticism by Members of the Opposition and the media about missing information in the business plans concerning performance measures and targets and the fact that performance measures were removed or replaced without any explanation. 47Although performance measures still get removed and baselines and targets are still being developed for some measures, one civil servant argued, "We are getting to the point where the measures are a lot more consistent and we are not changing what we are measuring every year.We have finally landed on what the important things are and we are going to measure these.We now have no hesitations in saying "this isn't really doing anything let's drop it" because we want the best measures possible. 48The criticism regarding changing measures is that it makes following a government's performance difficult if there is nothing to compare it with and it therefore makes it difficult to hold them to account.
Finally, and perhaps the most controversial within the Alberta legislature, are the challenges concerning the development of appropriate targets for each of the performance measures.Noting the political environment to which the measures and targets were being developed, one of the civil servants stated that there was a "fine art to choosing the target." 49For example, if a department is consistently meeting the stated targets, it would be important to raise the bar or you would risk being viewed as not being a risk-taking department.At the same time, it was important "to not do 'pie in the sky' either." 50Another civil servant articulated the complexity of choosing the "right" target: A department has to develop a formula and do it the best you can but to suggest it is flawless or easy, would be foolish.You don't always achieve.We didn't achieve in our department in one or two areas but we set our goals high, because you have to work harder to get to that goal, rather than to set your benchmarks or goals at low level and then achieve them every year. 51ny of the civil servant interview respondents noted that the many issues surrounding targets sparked the most debate within the internal business plan discussions.Further, the interview participants who were government members and media stated that the targets seemed to get the most attention in the press, legislature, and committees.
External performance reporting in government: subjective truth Not only are there challenges to developing, implementing, and evaluating performance measures in a technical sense, but also there are concerns about the fundamental challenges of publicly reporting in a political environment.Indeed, publicly reporting on a government's performance on an annual basis is a risky business in a political environment.Paul Thomas argues, "poor reports can damage ministerial reputations and negatively affect the position and resources of departments and programs.In short, there are risks involved with the collection and the publication of performance information." 52For example, the media, the Opposition, interest groups, think tanks, and citizens currently not in favour with the government in power may not be tolerant of negative results.Thomas further comments: When performance reports flow into the wider political area, the focus is most likely to be on the deficiencies of performance rather than providing a balanced picture.Opposition parties in legislatures have a stake in denigrating the performance and reputation of governments.Moreover, members of the public derive their impressions of government performance from the kaleidoscopic images provided in the mass media, based on "the horror stories" which are featured so prominently, the public concludes that nothing works.These are not "problems" for which there is a "managerial solution," rather they are "conditions" of political life that would have to change for performance measurement to work in the idealistic way that is intended. 53nce, there is a tension between the ideals of a performance measurement system in government and the political environment in which performance measurement is embedded.
In this type of politically charged environment, there is little room for mistakes in governing.While new public management has encouraged "continuous learning" and "learning organizations," 54 these concepts tend to be more tolerated internally than externally.For example, Members of the Opposition may not agree with the current performance measures selected to assess performance, the number of performance targets not being met, and the number of measures that change on an annual basis so that it is difficult to compare from one year to the next.While a government can argue that the process is relatively new and that they are "continuously learning" how to improve performance measurement processes and structures, as observed by Thomas, those external to government tend to be more critical than forgiving.Producing mass communication messages to citizens in whatever form can be challenging, but it is deemed to be a necessary feature in a liberal democracy.Dennis McQuail argues, "effective decision-making in a liberal democracy requires that citizens have access to all the information they need to evaluate the conduct of incumbent governments, to judge the merits of competing candidates for public office, and to assess the case for and against particular policy options." 55The Government of Alberta's mass communication messages are partially relayed via the business plans, annual reports, and the Measuring Up reports -all of which contain reporting on the government's performance.While the information provided in these documents is deemed to represent the goals, strategies, and measures of the Alberta Government, the plans and reports may not tell the "entire truth" given the political ramifications of negative performance.Paul Thomas further elaborates on his earlier point in this section: Ministers will seek to avoid the publicity and controversy that "bad news" brings -reacting defensively when something goes wrong.For their part, opposition parties can usually be counted on to interpret mistakes and shortcomings in performance in the worst possible light.When such clashes take place and are reported in the media, the issues involved become amplified and distorted.The whole process contributes to the public's impression that nothing or little in government works as intended. 56 related to the subject of this paper, citizens may then question the authenticity and accuracy of any government document if the perception is that "government doesn't work." In the interviews conducted with the media and Members of the Opposition, the overall theme was that the Government of Alberta only provided information that portrayed the government in a positive light.For example, a Member of the Opposition questioned the use of linking the business plans and performance measures to the goal of improving accountability: If they really wanted to make their government more accountable, they would have done it.I think it was much more a controlled public relations exercise to say to the public "look at how we manage and aren't we wonderful?"For the opposition, we used the business plan as a tool to measure, "Okay, you said you were going to do these 295 things and you only did 23 of them, what does this all mean?"So, for me, business plans became largely a tool we used in Question Period.In terms of being accessible to the public, I'm sure they are available to anybody that wants to read them, but I'm not sure anybody reads them.I'm not sure what impact the business plans have had; they were a way to give some more numbers.They were supposed to be a way to listen to the public.Do they really have an impact on policy?I really don't know. 57e of the questions that was asked during the interview process was for the interview participant to comment on the impact of the business plans and performance measures on the public.As noted above and as found in the interview results, there was little information about the level of knowledge the public had about these processes and to what extent these processes impacted citizens (e.g.improve accountability).For example, one civil servant notes, "I don't think the person on the street can understand what government is doing because the information is still too general.I don't think it will really hit home until it permeates the organization.I don't think most people are aware of our plans and measures." 58ted to one of the points made by the last interview respondent, another challenge of the business planning process is using the document to guide the ministry in its decisions and actions, which inadvertently questions the ultimate goal of performance measurement in the Government of Alberta.Most of the civil servant participants stated that their business plans still reflected only a small part of how decisions are made within the department.For example, one civil servant stated, The struggle that we are having in government is to start using the information.We have plans and we are gathering and reporting information about whether we are accomplishing those plans but it is not impacting the decision-making process at all.Does the government sit down and read each annual report to decide on whether to make a change to a different department and do they actually use these reports to make decisions about where to go to next?I don't see that happening.I see the exercise happening, a routine going on but I do not see an active integration with that information.It hasn't been available to the degree that it needs to be as still part of the transition.I don't see it happening the way it should.There is still resistance especially at the executive and senior management level.There is resistance to change -putting pen to paper and report on what you didn't achieve.That disclosure element -I don't see that happening in quite the open way that it is intended. 59ese comments by the interview participants show that there is still some cynicism and scepticism among those involved in the business planning process.As shown though, those who tend to give the more negative accounts tend to be those from the "outside looking in" -those from the Liberal and New Democratic parties and the media.
Internal incentives for telling the truth and nothing but the truth: another case of subjective truth?
The final area that challenges the authenticity and legitimacy of the performance measurement framework is the link between an individual's organizational performance targets and the Government of Alberta's performance management framework.In 1998, a performance management framework was introduced that rewarded and recognized employees for the results they achieved.The structural framework is the following: Department business plan goals link to the government business plan and cross-government priorities; performance goals and measures grow out of a department's business plan; employee performance measures align with those of the organization; and departments, teams and individuals are rewarded and recognized on the basis of these measures. 60ere are four components to the performance management framework: achievement of fiscal goals, achievement of business plan goals, demonstration of collaboration and support for corporate cross ministry policy initiatives, and demonstration and support for cross ministry administrative initiatives.While this topic could be a paper by itself, the importance of this link as it relates to this paper are the challenges of developing challenging targets when management's pay depends on the success of meeting those public targets.Further, the achievement bonus for managers is available only if government meets its debt reduction targets.This framework, upon first glance, is difficult to criticize as it provides an integrated and cascading approach to performance measurement.Yet the question that begs to be asked is to what degree will those responsible for developing targets establish measures and targets that are unattainable?Indeed, to be blunt, what is the incentive for staff to develop measures and targets that prohibit the management in the department from obtaining their full bonus?The Auditor General further comments on this conundrum: There may be an incentive for staff to set target levels conservatively to ensure achievement of the bonus.Conversely, the reviews need to consider that if the targets are unattainable, the amount of positive encouragement derived from the targets may be dramatically reduced. 61e probability that civil servants develop performance targets that consistently fail to meet the stated expectation are nil.While such unattainable targets may provide an incentive to improve services or programs, taking this approach may be what some have called a "career-limiting move."To take a public choice perspective, where political actors are deemed to act rationally, in a calculating fashion to maximize their utility, the development and maintenance of performance targets that are not met is then an irrational act.
Concluding thoughts: is propaganda better than no information?
While there is scepticism about the message being relayed through business plans and specifically, performance measures, there are still some positive attributes of such a performance system.As noted by the Government of Alberta, the intent of a public performance reporting system is designed to enhance citizens' understanding and support of public programs. 62There may be elements of truth in the business plans as they inform citizens, to a certain extent, what are the goals, strategies, and measures for each of the departments.At least citizens can be somewhat aware of what goals the Government of Alberta is pursuing and the results achieved in meeting those goals.Prior to 1993, the only information, for the most part, that could be found on each of the departments was in the annual reports, a results-oriented document that commented on a previous year's activities.
Moreover, the Alberta government argues that "a government that reports its own performance to citizens, rather than totally relinquishing that task to the media, has far more control over the manner in which information is disclosed and greater opportunity to describe its response to particular problems." 63Instead of citizens relying on the media to interpret government documents and announcements, by providing information to citizens themselves, the Government of Alberta can ensure that the message they want to convey is communicated in the manner they see fit, instead of through the media's interpretive lens.
One of the concerns raised by some of the interview respondents who were more sceptical of performance reporting was that performance measures were not integrated into the planning cycle and decision-making process in government.While the Government of Alberta seemed to have struggled with the integration of performance measures in departmental processes in the mid to late 1990s, there seems to be some improvement in this area in recent years.For example, the Standing Policy Committees and the Legislative Assembly are now using performance measures as part of their respective reviews of plans and budgets.Further, the Public Accounts Committee is using the new ministry annual reports to assess and discuss the performance of ministries.While these improvements have been made, the Auditor General explains that, "the linking of cost to results information is one of the remaining steps needed to assist users such as the Pubic Accounts Committee to effectively discharge their duties with respect to the review of public accounts." 64nally, perhaps the role of Alberta Auditor General diminishes the notion of business plans and performance measures as being propaganda tools of the Government of Alberta in their role as watchdog and critic.Further, the assessment of performance measures can also be conducted through feedback from citizens; feedback from the Government of Alberta, including Standing Policy Committees and Cabinet; questions or issues raised in the Legislative Assembly from the political parties not in power; and through departmental feedback via quarterly and annual reviews of departmental business plans.
There are few governments in the world that have developed a performance measurement framework as extensive, comprehensive, and integrated as the Government of Alberta.While this provincial government has been a leader in developing, implementing, and sustaining such a framework, as shown in this paper, there are still many technical and external and internal subjective challenges to overcome.Despite the technical challenges posed to any government developing a performance measurement framework, Paul Thomas argues that, The most appropriate stance to adopt on performance measurement is realism about its potential and its problems.This will involve steering a reflective and practical, middle course between naïve faith in rational techniques and the cynical use of performance measurement for purely symbolic purposes.It is my impression that most public servants strive to find this difficult, middle ground between commitment and cynicism. 65 demonstrated in this paper, publicly reporting on performance in a highly political environment is a complex task.On one hand, the goal is to provide citizens with information to enhance transparency, openness, and perhaps improve accountability.On the other hand, if the government only communicates "good news" or relays information that is not too damaging to a government's power base, and finally links management's salary/bonuses to achieving targets as outlined in the business plans, as shown by the interview results, it becomes difficult to seriously take this type of reporting as being credible and valid.Further, the technical challenges to performance measurement may further reduce the trust citizens have in the measures themselves.Yet the dilemma that remains is determining if indeed, it is possible for a government to produce information about their performance without the air of propaganda being applied to the dissemination and interpretation.Indeed, Donald Kettl argues that "performance based management is most fundamentally about communications, not measurement." 66