In your original thread discuss the following: Statesmanship as it relates to public administration reform and the future.The challenges and opportunities that a would-be statesman would face in this

New challenges for performance measures Deborah Agostino & Michela Arnaboldi To cite this article: Deborah Agostino & Michela Arnaboldi (2015) The New Public Management in hybrid settings: New challenges for performance measures , International Review of Public Administration, 20:4, 353-369, DOI: 10.1080/12294659.2015.1088686 To link to this article: https://doi.org/10.1080/12294659.2015.1088686 Published online: 17 Dec 2015.Submit your article to this journal Article views: 757View related articles View Crossmark dataCiting articles: 7 View citing articles The New Public Management in hybrid settings: New challenges for performance measures Deborah Agostino *and Michela Arnaboldi Department of Management Engineering, Politecnico di Milano,Via Lambruschini 4b, Milano, Italy (Received 30 November 2014; revised 18 February 2015; accepted 6 June 2015) This study questions whether Performance Measurement Systems (PMSs) defined under the New Public Management (NPM) logicfit the needs of hybrid settings where inter-organizational relationships exist between multiple actors in charge of service delivery. After outlining the key characteristics of an NPM-based PMS and its limitations in hybrid settings, an exploratory case study was carried out on a pub- lic network in charge of delivering a local public transport service. The network was particularly appropriate, as it initially endorsed an NPM-based PMS. Findings show problems in using the PMS and adapting the model, which led to the develop- ment of new features and requirements. Using a theoretical triangulation, it was possible to draw a more general insight into the characteristics of a PMS in a hybrid setting, here called a relational and participative PMS.

Keywords:performance measures; hybrids; public networks Introduction This article explores the extent to which performance measurement, developed follow- ing the New Public Management (NPM) paradigm,fits the current needs of public ser- vices, which are seen as hybrid realities where public and private organizations interplay to deliver a service.

Over the past 20 years, the NPM (Hood, 1991 ) paradigm proliferated among public administrations, supporting management practices and tools, often from the private sec- tor, that had been introduced to improve public service delivery. Following the NPM logic, performance measurement systems (PMSs) have acquired a central role. The work of three decades of empirical and theoretical contributions highlights some properties of these PMSs, which use the input/output model (Pollanen, 2005 ) as their main reference. A set of metrics can be derived from this model, to include efficiency, effectiveness and equity, together with several methods of performance planning and control linked to the hierarchical structure of the organization. These metrics have been widely adopted by public organizations all around the world, with studies exploring PMS design, implementation processes, and uses (Andrews & Kouzmin, 1999 ; Diefenbach, 2009 ; Guthrie & English, 1997 ). More recently, the NPM has come under greater scrutiny, with some authors (James, 2001 ; Osborne, 2006 ; Wiesel & Modell, 2014 ) questioning its role in a modern environment that has changed from the mid 1980s when it wasfirst introduced. *Corresponding author. Email: [email protected] © 2015 The Korean Association for Public Administration International Review of Public Administration, 2015 Vol. 20, No. 4, 353–369, http://dx.doi.org/10.1080/12294659.2015.1088686 The main challenge of these modern realities has been acknowledged in the shift toward hybrid settings, where private and public organizations must interact and col- laborate in order to deliver public services (Christensen & Lægreid, 2014 ; Koppenjan & Koliba, 2013 ), in a form often known as public networks. This movement from a single public administration to a complex hybrid structure to provide a service also poses problems for PMSs, with their divergent views about their own importance in these new settings. While some studies search for new metrics with which to evaluate the hybrid structures (e.g. Agostino & Arnaboldi, 2013 ; Kenis & Provan, 2009 , Provan & Lemaire, 2012 ; Provan & Milward, 2001 ), others affirm that performance measure- ment alone is not sufficient in hybrid contexts with multiple actors and inter-organiza- tional relationships (e.g. Kenis & Provan, 2006 ; Romzek, LeRoux, & Blackmar, 2012 ).With the purpose of contributing to extant literature on PMSs in hybrid settings, this article investigates the extent to which performance measures defined under the NPM logicfit the needs of hybrid settings, exploring the limitations of an NPM-based PMS and the new requirements for measuring performance. We carried out an empiri- cal investigation of these issues within the hybrid context of a public network for a local public transport service in Italy, composed of local administrations working jointly with private and public service providers to deliver the transport service locally. This case was of particular interest, since it initially attempted to manage the network using a PMS developed following NPM principles. Through interviews conducted over a per- iod of three years, it was possible to explore both the original NPM-based system and its limitations and the new emerging requirements for measuring performance.

Results show that PMSs in hybrid settings are associated with new requirements, in terms of both metrics and methods, leading to a revised PMS, here called relational and participative PMS. This revised PMS expands the traditional NPM-based approach with additional units of analysis, performance metrics, and introduces a participative approach to data collection and auditing procedures.

The rest of the article is structured as follows. The main characteristics of an NPM- based PMS willfirst be described, together with the extant contributions on PMSs in hybrid settings. Then the methodology of research will be presented, followed by the results, discussing the limitations of an NPM-based PMS in terms of metrics and methods. Finally, the emergent relational and participative PMS in hybrid settings will be discussed in the concluding section.

NPM and performance measurement NPM is a term that emerged in the mid1980s in UK and refers to a set of techniques and management tools, often derived from the private sector, that are applied in the public sector, with thefinal aim of improving the efficiency and effectiveness of public services (Hood, 1991 ,1995 ). These private sector techniques are said to reinvent government by‘lessening or removing differences between the public and the private sector and shifting the emphasis from process accountability toward a greater account- ability in terms of results’(Hood, 1995 , p. 94). Following this definition and the existing public administration practices and studies in thisfield (e.g. Hood, 1991 ,1995 ; Lapsley, 2008 ), performance measures are at the heart of NPM.

Performance measurement is an approach where indicators are used to evaluate the range, level, and content of services to be provided (Hood, 1991 ), and it is associated with the need to introduce control over the output produced by public administrations.

Indeed,‘by quantifying goals and measuring whether they are achieved, organizations 354D. Agostino and M. Arnaboldi reduce and eliminate ambiguity and confusion about objectives, and gain coherence and focus in pursuit of their mission’(Verbeeten, 2008 , p. 427). It has been suggested that, through performance measurement, public organizations can enhance their plan- ning and control over resources, leading to better value for money and improved ser- vices for the public (Sanderson, 2001 ). Three decades of empirical and theoretical studies on NPM led to the emergence of PMS and its distinctive features.

An NPM-based PMS can be referred to as a collection of KPIs (Key Performance Indicators) centered on an input/output model in which responsibility is concentrated, assigning the KPIs along the hierarchical line within the organizational units, and adopting formal procedures. Following this definition, an NPM-based PMS can be explored in terms of two key dimensions: metrics and methods (see Table 1).

Thefirst key dimension of an NPM-based PMS is that of its metrics, which have been developed starting from an input/output model (Pollanen, 2005 ). According to this model, some inputs are required to carry out certain activities that lead, in turn, to some outputs, such as the delivery of a service. Input refers to the amount of resources used to perform a certain activity while output refers to the result of a transformation pro- cess. Starting from this input/output model, the metrics are composed of two elements:

performance dimensions and unit of analysis.

Performance dimensions comprise a set of KPIs, which explore different aspects of the input/output model in terms of efficiency, effectiveness, and economy, also known as the 3E’s (Jackson & Palmer, 1992 ). Efficiency measures compare the service output with the required resources in input. Effectiveness measures evaluate the outcome characteristics of the delivered service, while economy measures are related to the value of the input resources used. Bouckaert and Van Doren ( 2003 ) introduced a fourth E, equity, used to evaluate whether everybody has the same opportunity of accessing a public service of the same quantity and quality. Furthermore, following this input/out- put model, performance measures are calculated with reference to a specific‘object of control’, also defined as unit of analysis. This unit of analysis is the organizational unit, which is controlled by performance measures and usually corresponds to the organiza- tion itself.

The second key dimension of the NPM-based PMS is that of its methods, intended as the approaches used for planning targets and controlling achievements. These meth- ods include a hierarchical line of communication in terms of performance measures and auditing procedures, to verify compliance with the selected KPIs. On the one hand, a hierarchical approach to PMS consists in making use of the different organizational levels, going from the top down to the operational units, to communicate and control performance plans. This hierarchical approach has also been defined as‘top-down’, with reference to‘a dominant concern for enhancing control and upwards accountability’ Table 1. Characteristics of an NPM-based PMS.

PMS characteristics NPM-based PMS Metrics Unit of analysis Organization Performance dimensions Efficiency, effectiveness, economy, equity Methods Approach to performance planning and controlHierarchical Auditing approach Formalized International Review of Public Administration355 (Sanderson, 2001 , p. 297). By proceeding along the organizational hierarchical line, it is possible to implement performance measurements within public administrations. This approach has been derived to counter the ambiguity that can be generated when defining targets: unclear objectives and measures can undermine improvements to performance (Rainey & Steinbauer, 1999 ). In order to limit this risk, roles and responsibilities must be clearly defined. It has been stated that‘a main goal of the NPM and reinvention-style reforms was to introduce a strategic“clarity of task and purpose”to public organizations through a variety of organizational reforms’(Moynihan & Pandey, 2004 , p. 426). These reforms had implications on how PMS can assist in the implementation of a clear hierar- chical structure with defined roles and responsibilities in terms of performance planning and control.

On the other hand, starting from the need to communicate performance measures (Lapsley, 2008 ) in a transparent manner, auditing procedures have been enhanced to provide a further, formalized control of the data collected by public administrations.

The requirement for higher transparency led to the development of the second PMS property, that of auditing. Power ( 1997 )defines audit as the‘control of control’since ‘auditors began to experience a wave of formalized and detailed checking up on what they do’(Power, 1997 , p. 3). Accordingly, audit procedures affect not only substantive activities, or what it is calledfinancial audit (Power, 2003 ), but also internal processes and systems of control (Sanderson, 2001 ). Therefore, the diffusion of NPM approaches has been associated with the parallel development of auditing procedures to control the process of data collection and management, which, in turn, is useful in controlling the activities of public administrations.

NPM-based performance measures have been widely developed around the world, although the timing and nuances are different (e.g. Leishman, Cope, & Starie, 1995 ).

For example, the UK was among thefirst countries to endorse these practices in mid 1980s, while other countries only adopted the approach a decade later (e.g. Andrews & Kouzmin, 1999 ; Guthrie & English, 1997 ).

Nowadays, NPM and its theoretical pillars have come under greater scrutiny because of the changes to public organization settings, mainly associated with the diffusion of hybrid organizational structures (James, 2001 ; Osborne, 2006 ; Wiesel & Modell, 2014 ).

Hybrid settings and PMS Hybrid is an ambiguous term broadly defining a‘new phenomena produced out of two or more elements normally found separately’(Miller, Kurunmäki, & O’Learly, 2008 , p. 943). In public administration, the notion of hybrid has usually been associated to organizational structures: hybrids are organizational structures set between market and hierarchy (Powell, 1990 ; Thomasson, 2009 ). This hybridity is often generated by the collision between two different realms, the public sector and the business area. It has been said that: These organizations [the reference is to hybrid organizations] are expected to function like businesses: to be efficient, customer driven, and client oriented. Yet, they perform tasks that are inherently public. In other words, they are supposed to act as if they were situated in the private sphere, while at the same time remain within the public sphere. (Kickert, 2001 , p. 136) 356D. Agostino and M. Arnaboldi Hybrid structures, such as partnerships, alliances, networks, and collaborative relation- ships (Agranoff & McGuire, 2003 ), are associated with two key features: multiple orga- nizational actors and inter-organizational relationships. Thefirst distinctive element of hybrid structures is that there are multiple organizational actors. The wide diffusion of contract-based reforms promoted by the NPM to improve public services, has been associated with an increased fragmentation of the service provision sector. Conse- quently, hybrid structures involve many actors that differ in terms of their nature and professional roles. They include public and private organizations, as well as policy- makers and public managers. The presence of many actors of a different nature can potentially clash with the need to implement a clear hierarchical structure in the PMS, something that was a characteristic of NPM-based PMSs.

The second distinctive element of hybrid structures is that of their inter-organiza- tional relationships, which connect this multiplicity of actors. The diffusion of service contracts to promote competition has, however, led to an increase in service delivery fragmentation, and public services need to be delivered in an integrated way. In this respect,‘intense and innovative cooperative working among public, private and volun- tary providers is promoted as a way of replacing the existing fragmented and dispersed service provision’(Kurunmäki & Miller, 2006 , p. 88). Concepts such as collaboration and cooperation are recurrent in public administration literature relating to hybrid struc- tures, with a variety of studies exploring the formation, management, and dynamics of collaborative practices between multiple actors (e.g. Kim, 2006 ; McGuire, 2006 ).

In this growing context of hybrid settings, performance measurement is often over- looked because of the argument that, due to their collaborative nature, hybrids are not associated with the need for performance control. Output controls, which are upheld and promoted by the NPM, have been criticized as not being appropriate in these new settings (Kenis & Provan, 2006 ). Having said this, the issue of performance measures has been analyzed by some scholars with the main purpose of exploring the way to measure the success or failure of hybrid structures. These studies are interested in quantifying how hybrid structures are performing, with particular emphasis on service effectiveness (e.g. Provan & Milward, 2001 ) and determinants of such effectiveness, including network structure, context and functional characteristics (Provan & Milward, 1995 ; Turrini, Cristofoli, Frosini, & Nasi, 2010 ). In this respect, traditional measures as well as ad hoc network measures have been proposed (Mandell & Keast, 2007 ). For example, with a specific focus on collaborative networks, the paper by Mandell and Keast ( 2007 ) suggests to develop ad hoc network measures based on relationships alongside traditional output measures. Andrews and Entwistle ( 2010 ) investigated cross-sectoral partnerships to understand whether they positively affect service perfor- mance in terms of effectiveness, efficiency, and equity. Similar studies can be found for networks (Provan & Milward, 2001 ), partnerships (Lee & Yoo, 2012 ), and collaborative relationships (Ryan & Walsh, 2004 ).

Extant studies do not specify a single, unique position for the role of performance measures in hybrid settings. While some search for new metrics (e.g. Agostino & Arnaboldi, 2013 ; Kenis & Provan, 2009 ; Mandell & Keast, 2007 ; Provan & Lemaire, 2012 ; Provan & Milward, 2001 ), others overlook performance measurement, suggesting alternative approaches based on informal arrangements (e.g. Kenis & Provan, 2006 ; Romzek et al., 2012 ). The aim of this study is to contribute to extant literature on PMS in hybrid settings, by exploring whether NPM-based PMSsfit modern hybrid realities.

Specifically, two research questions are addressed: What are the problems of adoptingInternational Review of Public Administration357 NPM-based PMSs in hybrid environments? Which are the new requirements for perfor- mance measures in these hybrid settings?

Methodology This study is based on the qualitative methodology of case studies (Yin, 2003 ), deemed as the most appropriate (Eisenhardt, 1989 ), because of its ability of providing an under- standing of complex behaviors and relationships in social contexts (Denzin & Lincoln, 2000 ). A single case study (Denzin, 1978 ; Yin, 2003 ) was conducted for this research, analyzing regulatory documents, archival data and official documentation, interviews and media observations. The following sections will contain a description of the research setting and then provide details on the process of data collection and analysis.

Research setting The selected hybrid setting is the public network for local public transport, involving both public administrations and private organizations. The public network corresponds to the local regional area and was imposed by the Region ( here called the Daphne Region, for reasons of confidentiality ) to integrate local public services at regional level.

The hybrid structure to provide a public service comprises four different categories of actors: Region, Provinces, Municipalities, 1 and service providers. The Daphne Region is responsible for the general governance of the public network, and is in charge of strategic transportation planning. Provinces and Municipalities have a regula- tory role for the bus service, each in their area of competence (either the province or the municipality ). They prepare competitive tendering for their local area and manage the service contracts of each service provider. Transport providers are private and pub- lic companies in charge of delivering the public service. The network, overall, is com- posed of one Region, 11 Provinces, 11 Municipalities, and 33 service providers.

In 2008, the Daphne Region introduced a regulatory intervention to integrate ser- vice delivery throughout the region. In this regulation, the individuality of the single actors was recognized, but there was also the requirement for them to operate jointly to deliver an integrated service in terms of interconnections, tariffs, and time scheduling.

Central to this reforming process was the introduction of a Performance Measurement System (PMS) to plan, manage, and control the activity of the different players, in order to deliver a single integrated offer to users.

Data collection and analysis Data were collected over a period of three years ( from 2009 to 2012) from a number of sources. Thefirst source of data involved regulatory documents. European, national, and regional laws on local public transport were reviewed, with the objective of build- ing a comprehensive picture of the national and regional transport landscape. This analysis was particularly useful in understanding the NPM settings in which the PMS was initially developed.

The second source included archival data in the form of performance reports, minutes of regional meetings, service contracts, and annual reports from all the stakeholders involved in the public network. The third source of data concerned the interviews with managers of transport provider organizations, managers, and politicians from public 358D. Agostino and M. Arnaboldi administrations and representatives of user groups. Each of the 28 semi-structured inter- views lasted between 45 and 110 minutes, and were recorded digitally and then tran- scribed. Interviews werefirst carried out with two representatives from the Region, then with eleven mobility managers of the Provinces and Municipalities, andfinally, with 12 transport company managers and three user group representatives. Interviews were par- ticularly useful in understanding characteristics, limitations and further developments of the implemented PMS.

This study has also drawn on media commentaries about the regional reforming process, with particular reference to the last regional reform of 2008. The media analysis, mainly from newspaper and television sources, reveals user concerns about their transport services. All the various sources of data were analyzed using a qualitative approach, which involved a textual analysis, sorting data into themes and cross-referencing them with theoretical principles in order to increase the internal validity of the case study material (Denzin, 1978 ).

Results This section discusses the NPM-based PMS developed by the Daphne Regional Administration to manage and control the hybrid setting of the local public transport network, and to address any problems in this system and the newly emerging require- ments. The PMS is discussed with a distinction made between metrics and methods.

The initial system will be analyzed for each, followed by an investigation of its limita- tions and exploring the new requirements.

PMS metrics A traditional NPM-based PMS was initially developed by the Region to cope with the need of ensuring an integrated transport service, while, at the same time, taking care of its demand to evaluate the contribution of each single actor involved in the service delivery. This section discusses the metrics involved in this initial PMS.

From a metrics perspective, the PMS followed the traditional input-output model, with a set of Key Performance Indicators (KPIs), grouped into efficiency, effectiveness, and equity, to be collected from each organizational actor in the system. Table 2syn- thesizes this set of metrics.

In line with the need to control output promoted by the NPM, the Region empha- sized the importance of transport service quality and, therefore, the system was weighted in favor of its effectiveness. Effectiveness was measured by quantifying ser- vice level, regularity, punctuality, comfort, safety, passenger information, cleanliness, and the perception of quality. Efficiency was measured in terms of productivity, while equity considered ticket cost, as this is linked to accessibility to the service on the part of different categories of users. As an example, service regularity at regional level was calculated by taking the average of the regularity metrics of all the service providers in the network. This system, which is aligned to the traditional NPM approach, was enhanced to suit the network reality. In the attempt to adapt the PMS to the new hybrid setting, the Region introduced an additional unit of analysis, the entire network, with the purpose of clearly distinguishing between the responsibilities of individual actors and network results. When the metrics for each provider were collected, the Region calculated the network value by summing or taking the average of these organizationalInternational Review of Public Administration359 data. As an example, service regularity at the network level was calculated by taking the average of regularity metrics for all service providers involved in the network itself.

The Region felt that this two-fold unit of analysis, at network and organizational level, was able to capture in a single dashboard both the performance of individual actors and the whole regional performance, since organizational data were combined.

Moreover, the Region believed that the unit of analysis of the single organization helped to identify specific responsibilities: given a certain result for the whole network in terms of performance, it was immediately possible to evaluate the contribution of each actor to the entire network value. Limitations to these NPM-based metrics did, however, appear when the system became operational within the hybrid structure.

Thefirst limitation appeared when the Region wanted to understand the benefits and problems in such joint working and the impact of collaboration on the network results. They then realized that there were no indicators in place to monitor the actors’ relationships and cooperation. By introducing two units of analysis, at organizational level and at whole network level, the Region’s initial ambition was to encourage the spontaneous cooperation between the transport companies, in order to improve regional transport performance. In practice, this voluntary cooperation did not materialize and, on the contrary, service providers were in competition with each other, showing no willingness to share their data or information with other providers, as emerged from this interview:

Table 2. Set of NPM-based metrics.

Performance dimension Description KPI Efficiency Productivity Bus-km Cost per bus-Km Effectiveness Service level n. of municipalities receiving the service average days of service n. of service lines Regularity % bus journeys not provided % bus journeys ahead of the timetable Punctuality % of bus journeys under 3 minutes % of bus journeys under 5 minutes % of bus journeys under 10 minutes Comfort n. of bus stops with bus-shelters n. of bus shelters with benches n. of buses with air conditioning n. of buses with provisions for disabled passengers Safety n. of buses with video cameras n. of buses connected to central control PassengersInformation n. of buses with information n. of bus shelters with maps n. of bus stops with timetables Time it takes for call center to answer Cleanliness Frequency of bus cleaning service Frequency of bus shelter cleaning service Quality perception Customer satisfaction Equity Tickets Cost of single, weekly, monthly annual tickets 360D. Agostino and M. Arnaboldi Why should we share our information about the service with other providers? This data is useful for us, to improve our own operations (General Director–Transport Provider A). This lack of cooperation also gave rise to operative problems about service provision.

A minimum level of cooperation was required to simplify the process of establishing interconnections between the transport services of different providers, but this did not always occur. The General Director of one of these operators stated that: In order to align my timetable to that of the other providers, I asked them for their timeta- ble for that specific transport node. It’s important that passengers on my bus-line arrive in time and don’t lose their connection because our timetables don’t match up. No timetable has turned up as the other providers don’t want to share their data with me. In the end, I was forced to check the timetables on their websites, and even then, they were out of date (General Director–Transport Provider C). This scant cooperation underlines thefirst limitation of an NPM-based PMS, since it was clear that the status of the relationships between the network actors could not be understood by simply measuring performance through units of control at organizational and network levels only. In order to overcome this limitation, the Region revised the initial PMS and added an additional unit of analysis, that of relationships. In this way, metrics relating to the actors’relationships were also collected. Metrics about the quan- tity and quality of relationships were introduced to check whether the network actors were cooperating. This issue is connected to the recognition that a network can be good in terms of activities, but perform poorly in terms of results, and vice versa (Kenis & Provan, 2009 ).

Strictly correlated to this aspect, the limitations in the measures of efficiency, effec- tiveness, and equity were also acknowledged. The Region introduced three perfor- mance-related aspects that it found useful to support network decisions. For example, data about service punctuality were used to establish the increase in network target levels every year. However, during the interviews, it came out that the metrics devel- oped initially did not provide the information to monitor the most significant aspect of the network that of resource integration, understood as the available network resources in terms of the actors’interconnections. Accordingly, resource integration was an aspect added to the PMS. Resource integration means evaluating whether separate elements can be incorporated into a whole. With reference to the local public transport service, integration was measured in terms of number of integrated tickets, customer perception about the level of integration, and also the network investment in interconnections and network personnel.

It followed that the NPM-based PMS was revised from its initial version and became more relational in nature, with the addition of the unit of control of relation- ships and the performance dimension of resource integration.

PMS methods In this section, the NPM-based PMS will be analyzed in terms of methods, which include several approaches to planning and controlling performance, and auditing procedures.

With respect to performance planning and control, a hierarchical approach was introduced. This hierarchy started from the Region, and then moved down to Provinces and Municipalities, andfinally to the service providers. The process of performance planning and control followed a specific pathway. The Region set the metrics and International Review of Public Administration361 standard values that were formalized in service contracts between Provinces or Munici- palities and service providers.

The path was then reversed, going from providers to Provinces and Municipalities, and back to the Region during the operations relating to control, where annual perfor- mance reports were prepared by the providers and, following the reversed hierarchy, sent to the Region.

The Region was at the centre of this planning and control system, as it decided the type of metrics to collect, structure of the reports and associated informationflow.

Once in place, this centralized and hierarchical system was then criticized by the service providers, who complained about their lack of involvement. Some providers opposed the hierarchical approach of the regional managers in defining the indicators to be included in PMS. In this respect, one of them argued that: They selected the measures without asking us–the people in charge of delivering the ser- vice from an operational point of view - which indicators would have been useful in managing service provision. This is like a restaurant menu, where everything must be veri- fied subjectively (CFO–Transport Provider B). Other providers further criticized the Region’s approach, highlighting their reaction to the hierarchical approach: We have gone from having no control and almost complete freedom in managing our operations, to control over everything! We just received these metrics without being ini- tially involved in their definition. (CFO–Transport Provider D). These complaints prompted a lively debate between the providers, and also between the providers and the Provinces or Municipalities and the Region, with the providers’ aim of changing or revising the data initially included. Initially, the dialogue was more a dispute because it was centered on complaints about performance measures. It then moved to a more balanced practice of interacting and sharing opinions before making performance-related decisions, gradually transforming the hierarchical approach to per- formance planning and control into a more participative practice.

Concerning the auditing procedures, the initial NPM-based PMS was highly formal- ized. The Regional regulation formally assigned to Provinces and Municipalities the role of auditing performance measures from service providers on a yearly basis. This audit process was conceived within the hierarchical structure of reporting, since perfor- mance measures were collected annually by the providers, transmitted to the Provinces and Municipalities, who had the role of auditing the data and transmitting them to the Region, who then elaborated the overall network picture.

This formal auditing role of the Provinces and Municipalities, once in place, showed its limitations, as attested by the Region itself, and also by the service users.

One Regional representative referred to the Provinces and Municipalities as‘paper- pushers’(interviewee’s words). He said: Service regulators should verify the data they collect from service providers. Instead, they simply collect these measures and transmit the reports to us. This is their non-role. After that we saw some very good performance data, while, at the same time, we received a number of complaints from users about the same performance. You then start questioning the role of the Provinces and Municipalities (Transport Unit Administrative Director– Daphne Region). A more severe criticism about performance auditing came from transport service users, one stating: 362D. Agostino and M. Arnaboldi Why should I even consider this report? [Referring to published data] The service provider is self-certifying its work. What can I say? That it is delivering an almost perfect service?

I don’t think I canfind reliable information. I prefer to take note of the complaints and warnings I receive every day from passengers (User Representative A). This quote highlights the complaints of users, and also their reaction to the limitations of the current system. To overcome the problem of data reliability, users started audit- ing the performance results of service providers in an informal way. They used the real-time data collected daily by passengers and shared through emails and social chan- nels. A user association was in charge of gathering the travel warnings, forwarding the data periodically to the Region. These emails ( pointing at his laptop screen) are today’s transport problems. Passengers tell us about the problems they face when using local public transport, from the air-condition- ing that is not working to bad punctuality on the tram system, to the poor condition of a bus-shelter. We track all this information and use it in our internal transport analysis (General Director–Users’association). This practice resulted in moving the auditing process from formal to informal and user-driven, leading to still another revision of the NPM-based PMS. From a methods perspective, the initial PMS was reconfigured to introduce wider participation in the performance planning and control process, and informal auditing procedures based on the involvement of network actors and service users.

From an NPM-based PMS to a relational and participative PMS The empirical analysis showed the attempt on the part of the Region to develop a PMS for the hybrid structure of the network, following the traditional NPM-based paradigm.

Accordingly, the metrics comprised a set of KPIs grouped into the performance dimen- sions of efficiency, effectiveness, and equity, which were collected using the service providers as unit of analysis. The unique effort of adapting the PMS to a hybrid struc- ture was visible at the control unit level, with the introduction of the network unit of analysis. This choice made by the Region was an attempt to apply the NPM principle of clearly identifying the centers of responsibility, and the relationships between them, to the network. Indeed, the intention, at the network level, was to control both the per- formance of each actor and the overall network service. However, this NPM-based PMS highlighted its limitations once it became operational. The difficulty of using the system to evaluate the actors’cooperation and the network resources called into ques- tion the validity of the PMS. The main problem was that the NPM-based system did not account for the distinctive features of hybrid structures with their multiple actors and inter-organizational relationships. These aspects led to the NPM-based PMS evolv- ing toward a revised system, which we have defined here as a relational and participa- tive PMS. This revised PMS differs from the NPM-based system in terms of both metrics and methods.

Thefirst revised aspect relates to metrics. The relational and participative PMS involves the introduction of the additional unit of analysis of relationships and the per- formance dimension of resource integration. Using the unit of analysis of relationships, it was possible to monitor performance at the relational level, and this improved control over the interactions between the network actors, a crucial aspect in terms of the network delivering the transport service. This appeared particularly useful in mandated networks, such as the network being analyzed here, where there was only limitedInternational Review of Public Administration363 willingness among the actors to cooperate. Furthermore, the revised PMS also included the additional performance dimension of resource integration, with indicators about the available resources for the network, such as personnel or investments for interconnec- tions. The importance of adding measures about resource integration derives from a dis- tinctive feature of inter-organizational relationships, whereby delivering a joint output can support more effective control of network activities and the actors’interconnec- tions.

The second distinctive feature of the relational and participate PMS concerns its methods. The NPM-based PMS initially relied on a hierarchical and formalized approach to performance planning, control, and auditing. This approach, however, showed its limitations when applied to the hybrid organizational structure of the net- work. It proved impossible to manage all the different actors, in the form of public administrations and service providers, through a formal hierarchy among the many par- ties, even in the case of a mandated network. On the contrary, the emergent relational and participate PMS involved a participative approach between the network actors, which meant including not only public administrations and business companies, but also service users, in the process of PMS planning, control, and auditing procedures.

The involvement of users led naturally to a further change to the auditing procedures, which became more informal, based on the users’individual experience shared through emails and social channels.

This relational and participative PMS was not defined top-down a-priori, but emerged autonomously from the network actors who tried to overcome the limitations of the NPM-based PMS introduced initially. Table 3compares the NPM-based PMS with the emergent relational and participative PMS.

Conclusions This study investigated thefit of NPM-based PMS in the hybrid setting of public net- works, by exploring the limitations of performance measures developed under the NPM paradigm and the new emergent requirements for measuring performance in these hybrid settings.

The study started by analyzing the distinctive features of an NPM-based PMS, which has been described as a collection of KPIs (Key Performance Indicators) cen- tered on an input/output model, to be applied in organizations where the hierarchical structure is clearly defined (Hood, 1991 ; Jackson & Palmer, 1992 ; Pollanen, 2005 ; Verbeeten, 2008 ). From this definition, the PMS has been analyzed following its two main dimensions of metrics and methods. Table 3. NPM-based PMS and the revised relational and participative PMS.

PMS characteristics NPM-based PMSRelational and participative PMS Metrics Control unit Organization and whole systemRelationships Performance dimension Efficiency, effectiveness, equityResource Integration Methods Approach to performance planning and controlHierarchical Participative Auditing approach Formalized Informal 364D. Agostino and M. Arnaboldi The methodological approach of the case study has been adopted by investigating a local public network in charge of delivering a transport service. This network intro- duced a PMS following the NPM principles, but problems were encountered once the system was in place, which led to a further revision of the system itself. The analysis of the limitations of the PMS introduced, and the emergence of new requirements, lead to the identification of a revised PMS for hybrid settings, here called relational and par- ticipate PMS. This new PMS is relational from a metrics perspective, given that it accounts for the unit of analysis of relationships and the performance dimension of resource integration, which both support the management and control of multiple actors and their interconnections. This revised PMS is also participative in the process of per- formance planning, control, and auditing procedures. Indeed, the simultaneous partic- ipation of the actors involved in the hybrid structure spontaneously arose during all the activities related to performance management from planning, implementation and, finally, auditing. Actors involved comprise, not only public administrations and busi- ness companies, but also every day users, giving rise to a participative approach to per- formance management. Indeed, when the audit process is managed with the contribution of everyday users, this has several implications also in terms of power issues within the network: rather than having a clear hierarchical line with a central unit of control, the PMS process becomes decentralized and user-driven.

Thesefindings provide contributions for both academic and practitioners. From an academic perspective, three main areas are enhanced. Thefirst one concerns the identification of new requirements for PMS in hybrid settings. Starting from the limita- tions of the NPM-bases PMS, this study proposes a relational and participative PMS in order to betterfit the new reality of hybrid structures characterized by multiple actors and inter-organizational relationships. The challenge of this PMS with respect to an NPM-based system lies in both measures and methods. On the one hand, the revised PMS is suggested to evaluate the unit of analysis of relationships and to quantify the level of resource integration. This variation is required in order to control interactions among actors and their willingness to cooperate, an aspect that was ignored by the previous system. On the other hand, the revised PMS is characterized by a participative approach to the entire process of performance planning, control, and auditing. This fur- ther revision enhances the actors’motivation to be part of the greater whole of the net- work, prompting their active collaboration in network management and control. The proposed relational and participative PMS contributes to extant literature on perfor- mance measures in the public sector, moving ahead from an NPM-based paradigm that was well suited in a public administration context characterized by the centrality of public administrations (Hood, 1991 ; Verbeeten, 2008 ), by suggesting ad hoc measures and methods tofit the new reality of hybrid structures. The proposed PMS supports the idea that the specific features of network relationships demand ad hoc measures, which move beyond service effectiveness, by including the ability to establish relationships and share resources, further supporting the position by other authors (e.g. Mandell & Keast, 2007 ). Furthermore, this study contributes to the literature specifically focused on measures for hybrid settings (e.g. Provan & Milward, 2001 ), suggesting that mea- sures do matter, but for hybrid settings to work, these measures need to be associated with a participative approach to performance management, control, and auditing.

Specifically, the proposed PMS underlines the importance to move away from top- down network evaluation by endorsing participative approaches based on actors’ involvement in order to obtain these new network measures. This participative approach can provide benefits, not only in terms of reliability of the collected data, but International Review of Public Administration365 also in terms of network functioning by further stimulating actors interacting with each other and therefore reinforcing existent network relationships.

The second academic contribution recognizes the importance to manage and control hybrid structures through a PMS, rather than relying on informal approaches only. This aspect is not obvious, given that the relevance of a PMS has been questioned by some studies (Kenis & Provan, 2006 ; Romzek et al., 2012 ), which considered performance measurement not adequate in these new organizational arrangements. The present study suggests that PMS does matter in hybrid contexts in order to support network manage- ment and actors’motivation, but it should be revised by considering the specific fea- tures of multiple actors and inter-organizational relationships.

The third and last contribution for academic literature is related to the growing importance of service users within these new hybrid organizational structures in charge of public service delivery. Users, rather than being passive receivers of public services, have emerged to play an active role in performance management by auditing network actors’data on the basis of their everyday experience with the public service. Thisfind- ing contributes to extant public administration literature that recognizes the centrality of service users (e.g. Liao & Zhang, 2012 ; Nam, 2013 ) by including their active role in the process to audit service providers’reports and network data. The increasing impor- tance of users might have potential impact also on power relationships within hybrid structures, by altering the equilibrium among the involved actors. Further research can explore the impact of users’involvement in hybrid structures and associated power dynamics.

From a practitioner perspective, this study provides managers of hybrid structures with some practical details on how to design a PMS for these organizational structures.

The relational and participative PMS suggests that traditional measures to quantify effi- ciency and effectiveness of public services must be complemented with ad hoc metrics to quantify relationships, intended as the ability of the actors involved to exploit their activity in collaboration with the other partners, and the integrated resources devoted to these new settings. Furthermore, suggestions are provided also in terms of managing the new relational and participative PMS: managers of hybrid structures are recom- mended to involve other actors in charge of service delivery and service users, giving rise to a participative approach to the process of PMS design, and then auditing these practices once implemented. The reasons behind these changes in both metrics and methods that characterize the relational and participative PMS lies in a better support to both decision-making and motivation within hybrid settings. Indeed, while in the past the central role of public administration was required to manage public services, this assertion does not hold in hybrid structures, where public administrations also become one actor within a greater whole. Empirical evidence, furthermore, suggests that, even without revising the traditional NPM-based PMS, the system autonomously evolves toward a more relational and participative system. Some additional studies are required in order to explore the validity of thesefindings in other hybrids; we focused on public networks, but some other research could apply the same analysis to other hybrid struc- tures, such as partnerships or alliances.

Note 1. In Italy, in terms of administration, central government is organized locally into Regional adminis- trations, further divided into smaller territorial areas called Provinces and Municipalities.

366D. Agostino and M. Arnaboldi Notes on contributors Deborah Agostinois Assistant Professor at Politecnico di Milano, Department of Management, Economics and Industrial Engineering. Her research interests cover mainly two areas. Thefirst area is related to performance management in public networks, with a specific focus on networks for the provision of the local public transport. The second area of investigation covers perfor- mance measurement systems in the social media era. Her publications can be found inPublic Management Review,European Management JournalandPublic Relations Review. Email:

[email protected] Michela Arnaboldiis Full Professor at Politecnico di Milano, Department of Management, Eco- nomics and Industrial Engineering. Her research activity is mainly focused on two areas: manag- ing performance in the Public Sector and the evolution of Management Control Systems. This second issue has been carried out along two dimensions: the analysis of the relationship between budgeting system and enterprise risk management; the investigation of the role of management control system in private-public networks. Some of her publications can be found inAccounting Organizations and Society,Financial AccountabilityandManagement Accounting Research.

Email: [email protected] References Agostino, D., & Arnaboldi, M. (2013). How Performance Measurement Systems support manage- rial actions in networks: Evidence from an Italian case study.Public Organization Review:

Advance online publication. doi: 10.1007/s11115-013-0264-5 Agranoff, R., & McGuire, M. (2003).Collaborative public management: New strategies for local governments. Washington, DC: Georgetown University Press.

Andrews, K., & Kouzmin, A. (1999).‘Naming the Rose’: New public management discourse in the Brazilian context.International Review of Public Administration, 4,11–20.

Andrews, R., & Entwistle, T. (2010). Does cross-sectoral partnership deliver? An empirical exploration of public service effectiveness, efficiency, and equity.Journal of Public Administration Research and Theory, 20, 679–701.

Bouckaert, G., & van Doren, W. (2003). Performance measurement and management in public sector organisations. In T. Bovaird & E. Lofler (Eds.),Public management and governance ( pp. 151–164). London: Routledge.

Christensen, T., & Lægreid, P. (2014). Performance and accountability—a theoretical discussion and an empirical assessment.Public Organization Review: Advance online publication.

doi: 10.1007/s11115-013-0267-2 Denzin, N. K. (1978).The research act: A theoretical introduction to sociological methods.

New York, NY: McGraw-Hill.

Denzin, N. K., & Lincoln, Y. S. (2000). The practices and politics of interpretation. In M. K.

Denzin & Y. S. Lincoln (Eds.),Handbook of qualitative research( pp. 897–922, 2nd ed.).

Thousand Oaks, CA: Sage Publications.

Diefenbach, T. (2009). New public management in public sector organizations: The dark sides of managerialistic‘enlightenment’.Public Administration, 87, 892–909.

Eisenhardt, K. M. (1989). Building theories from case study research.Academy of Management Review, 14, 532–550.

Hood, C. (1991). A public management for all seasons?Public Administration, 69,3–19.

Hood, C. (1995). The‘New Public Management’in the 1980s: Variations on a theme.

Accounting, Organizations and Society, 20,93–109.

Jackson, P. M., & Palmer, B. (1992).Developing performance monitoring in public sector organizations( pp. 1–35). The Management Centre: University of Leicester.

James, O. (2001). New public management in the UK: Enduring legacy or fatal remedy?

International Review of Public Administration, 6,15–26.

Kenis, P., & Provan, K. G. (2006). The control of public networks.International Public Management Journal, 9, 227–247.

Kenis, P., & Provan, K. G. (2009). Towards and exogenous theory of public network performance.Public Administration, 87, 440–456. International Review of Public Administration367 Kickert, W. J. M. (2001). Public management of hybrid organizations: Governance of quasi-autonomous executive agencies.International Public Management Journal, 4, 135–150.

Kim, J. (2006). Networks, network governance, and networked networks.International Review of Public Administration, 11,19–34.

Koppenjan, J., & Koliba, C. (2013). Transformations towards new public governance: Can the new paradigm handle complexity?International Review of Public Administration, 18(2), 1–8.

Kurunmäki, L., & Miller, P. (2006). Modernizing government: The calculating self, hybridization and performance measurement.Financial Accountability and Management, 22,87–106.

Guthrie, J., & English, L. (1997). Performance information and programme evaluation in the Australian public sector.International Journal of Public Sector Management, 10, 154–164.

Lapsley, I. (2008). The NPM agenda: Back to the future.Financial Accountability and Management, 24,77–96.

Lee, S. J. & Yoo, D. S. (2012). The adoption of collaborative governance institutions: The EPA-States Performance Partnership Agreements (PPAS).International Review of Public Administration, 17, 143–161.

Leishman, F., Cope, S., & Starie, P. (1995). Reforming the police in Britain: New public management, policy networks and a tough‘old bill’.International Journal of Public Sector Management, 8,26–37.

Liao, Y. & Zhang, Y. (2012). Citizen participation in local budgeting: Mechanisms, political support, and city manager’s moderating role.International Review of Public Administration, 17,19–38.

Mandell, M., & Keast, R. L. (2007). Evaluating network arrangements: Toward revised performance measures.Public Performance & Management Review, 30, 574–597.

McGuire, M. (2006). Collaborative public management: Assessing what we know and how we know it.International Review of Public Administration, 66,33–43.

Miller, P., Kurunmäki, L., & O’Learly, T. (2008). Accounting, hybrids and the management of risk.Accounting Organizations and Society, 33, 942–967.

Moynihan, D. P., & Pandey, S. K. (2004). Testing how management matters in an era of government by performance management.Journal of Public Administration Research and Theory, 15, 421–439.

Nam, T. (2013). Citizen participation in visioning a progressive city: A case study of Albany 2030.International Review of Public Administration, 18, 139–161.

Osborne, S. (2006). The new public governance?Public Management Review, 8, 377– 387.

Pollanen, R. M. (2005). Performance measurement in municipalities: Empirical evidence in Canadian context.International Journal of Public Sector Management, 18,4–24.

Powell, W. W. (1990). Neither market nor hierarchy: Network forms of organization. In L. L.

Cummings & B. Staw (Eds.),Research in organizational behavior( pp. 295–336).

Greenwich, CT: JAI Press.

Power, M. (1997).The audit society: Rituals of verification. Oxford: Oxford University Press.

Power, M. (2003). Evaluating the audit explosion.Law and Policy, 25, 185–202.

Provan, K. G., & Lemaire, R. H. (2012). Core concepts and key ideas for understanding public sector organizational networks: Using research to inform scholarship and practice.Public Administration Review, 72, 638–648.

Provan, K. G., & Milward, H. B. (1995). A preliminary theory of network effectiveness: A comparative study of four community mental health systems.Administrative Science Quarterly, 40,1–33.

Provan, K. G., & Milward, H. B. (2001). Do networks really work? A framework for evaluating public sector organizational networks.Public Administration Review, 61, 413–423.

Rainey, H. G., & Steinbauer, P. (1999). Galloping elephants: Developing elements of a theory of effective government organizations.Journal of Public Administration Research and Theory, 9 (1), 1–32.

Romzek, B. S., Leroux, K., & Blackmar, J. M. (2012). A preliminary theory of informal account- ability among network organizational actors.Public Administration Review, 72, 442–453.

Ryan, C., & Walsh, P. (2004). Collaboration of public sector agencies: Reporting and accountability challenges.International Journal of Public Sector Management, 17, 621–631.

Sanderson, I. (2001). Performance management, evaluation and learning in model local government.Public Administration, 79, 297–313. 368D. Agostino and M. Arnaboldi Thomasson, A. (2009). Exploring the ambiguity of hybrid organizations: A stakeholders approach.Financial Accountability and Management, 25, 353–366.

Turrini, A., Cristofoli, D., Frosini, F., & Nasi, G. (2010). Networking literature about determinants of network effectiveness.Public Administration, 88, 528–550.

Verbeeten, F. H. M. (2008). Performance management practices in public sector organizations:

Impact on performance.Accounting, Auditing & Accountability Journal, 21, 427–454.

Wiesel, F., & Modell, S. (2014). From new public management to new public governance?

Hybridization and implications for public sector consumerism.Financial Accountability and Management, 30, 175–205.

Yin, R. K. (2003).Case study research: Design and methods. London: Sage. International Review of Public Administration369