zIndex - public contracting authorities rating

zIndex is a public procurement benchmarking tool for contracting authorities. It uses real data to measure each contracting authority's rate of transparency, efficiency and corruption potential in public procurement. In a nutshell, the zIndex measures the contracting authority's compliance with best practice recommendations defined by international organizations, the Czech Ministry of Regional Development, and other non-governmental organizations.

Best practice involves more than just proceeding in accordance with the law. Legal provisions inherently cover all sorts of marginal situations and exceptions, hence they establish rather broad rules of conduct. Best practice describes not only the behaviour legally required of contracting authorities, but the most desirable behaviour within those rules.

The purpose of zIndex is to highlight both controversial and exemplary contracting authorities. By evaluating different areas of contracting practice separately, it is able to identify areas for specific improvement. zIndex is the result of academic research undertaken at the Faculty of Social Sciences, Charles University in Prague. It is published at zIndex.cz by Econlab NGO. The EconLab's ambition is to further develop the zIndex methodology, publish evaluations of different authority subsets (municipalities, government departments, state-owned enterprises, etc.) on a regular basis, and in so doing, gradually increase the pressure on the authorities to improve their public procurement practice.

zIndex is calculated based on contract data spanning a period of three years. That timeframe is shorter than a standard electoral cycle, but still sufficient to create a representative data sample. This means that the political representatives in charge of certain institutions can be held accountable for those institutions' zIndex values before the elections come round again.

Despite being a relatively robust statistical indicator, zIndex cannot reflect all aspects of the ideal procurement process, especially when it comes to qualitative aspects. The zIndex score benchmarks the contracting authorities only according to objective, well measurable criteria. Given that those account for most aspects of the procurement process, this is a fair way of assessing public procurement.

A low zIndex rating implies a deviation from best practice (i.e. from good conduct, as defined by international institutions). Contracting authorities with low zIndex values (in comparison to similar institutions) are not necessarily more corrupt or less efficient; the low score simply indicates there is more room for corrupt or inefficient practices in their procedures - but whether that opportunity has been exploited or not cannot be proven from these statistics. Deviation from best practice is not always a bad thing, in some specific circumstances it might be beneficial for the procurer not to follow the best practice (for example it is reasonable to cancel procurement procedure if procurer realize that the contract documentation is incorrect). zIndex.cz web provides all contracting authorities with the opportunity to comment officially on their zIndex results and to explain their score, if they wish.

The zIndex project's long-term goal is to serve as an educational tool, using illustrative comparisons to identify contracting authorities' weaknesses, and to use the results to better educate contracting authorities in best practice and motivate them to better manage public funds. That is why, on the zIndex web pages, you will not only find descriptions of the individual sub-indicators we study and explanations of particular problems identified, but also practical improvement recommendations, references to useful literature, and other tools to help authorities improve their practice. The project is designed to provide a place for factual debate about appropriate public procurement practice, for contracting authorities, the public, and the media.

zIndex scores are compiled on the basis of nine partial indicators, each of which describes a particular public procurement issue. Together, these indicators cover most aspects of the procurement process. The resulting evaluation provides information on three main topics:

1. Accessibility - are the contracts accessible to bidders?
2. Competition - do several bidders really compete for each contract?
3. Supervision - are the details of the relevant cash flows publicly available?

Openness is a substantial prerequisite for fair public procurement procedure. If potential bidders are not informed about a call for tender and thus have no chance of bidding for it, the existing bidders are not under any pressure to offer a better quality/price ratio (a lower price or higher quality product). Procedures in which contracts are awarded without open competition are far more liable to corruption, patronage, cartel agreements and other unfair practices. For zIndex, we evaluate accessibility on the basis of three indicators:

1. Public procurement as a share of total purchases

2. Competitive contracting

3. Consistent conduct

Direct competition among potential suppliers is an essential part of public procurement, because it encourages lower prices and higher quality goods and services. A direct competition environment is also much less susceptible to corrupt practices or cartel arrangements. We evaluate the level of competition on the basis of the following indicators:

4. Winner concentration

5. Bidder participation

6. Pro-competitive tools

Last but not least, it is vital that public procurement contracts can be fully scrutinised, ideally both by government authorities and by the general public. We look at the opportunities for inspection and audit, based on the following three indicators:

7. Legal misconduct

8. Journal data quality

9. Buyer profile data quality

The nine indicators are combined to produce a single zIndex value, using a weighted average:

$$zIndex = \frac{z_1 + z_2+ z_3+ z_4+ z_5+ z_6+ z_7+ z_8+ z_9}{9}$$

where $z_{1}$ to $z_{9}$ are the values of the individual indicators, numbered as in the list above.

Our primary data source for the evaluation is the Czech Public Procurement Journal (hereinafter referred to as the Journal). Essential information about all public contracts procured in the Czech Republic in compliance with the Public Procurement Act 134/2016 Coll. (i.e. both „above-the-threshold“ and „below-the-threshold“ public contracts) is published in the Journal. We retrieve that information from the Journal using special software, then clean it and pair it with extra data, which we obtain from other public registers or from Czech Credit Bureau a.s. (CRIF). Other datasets used include information about donations to political parties, from the politickefinance.cz server; financial statements from the State Treasury depository; the decisions made by the Office for the Protection of Competition; data retrieved from buyer profiles; and information provided by the contracting authorities themselves in response to inquiries under the Freedom of Information Act. All the data is then processed, in cooperation with our affiliated company Datlab s.r.o.

• The methodology described here is suitable for benchmarking contracting authorities with similar structures and volumes of purchases. It is not suitable for comparing markedly different types of authorities, e.g. a municipality and a state owned enterprise. For institutions with unique spending patterns (such as the Road and Motorway Directorate), zIndex evaluation only has an indicative value.

• A procurement is usually considered to fall within the reference period if the relevant contract award notice was published within that period. Indicators which also analyse data on unawarded contracts (Consistent conduct, Legal misconduct, Pro-competitive tools) use other data to determine timing - typically the contract notice publication date. All exceptions are discussed under the relevant indicator.

• Due to possible irregularities in public data, partial zIndex results are provided to the contracting authorities involved in advance, so that possible corrections may be made and unnecessary reputation risks avoided.