Buyer profile data quality

Bidder participation is one of the indicators used to calculate the zindex score for evaluating contracting authorities.

This indicator evaluates whether and how contracting authorities publish contract information on their electronic buyer profiles, using a machine-readable interface that every profile is legally obliged to have. The following three aspects are evaluated:

• Functionality - does the profile have a machine-readable access as required by law?
• Completeness - does the profile contain information on all the contracts published in the Journal?
• Quality - does the profile contain any apparent errors or information inconsistent with the Journal?

Buyer profile is an obligatory instrument for publishing more detailed (when compared to the Journal) information about public contracts. That includes above all the tender specifications, contract agreements, exact prices actually paid, list of subcontractors etc. According to the best practice guides the use of electronic profiles facilitates contract publicity, accessibility and public control, as well as reduces administration burden and corruption risks.

All electronic buyer profiles of each contracting authority are surveyed using a web bot and results of this surveillance including the entry data are recorded. In case of inaccessibility several attempts to retrieve the data were made.

Information available must be consistent with the information advertised in the Journal, in terms of number of tenders, correct identification of both the contracting authority and the winning bidder or the state of the tendering procedure (contract awarded/cancelled etc.). The winning price must be closely similar (deviation of less than 1 %). Data quality is also examined - using elementary requirements such as „for every contract awarded there must be the respective winning bidder declared“.

If a contracting authority has (or at any time had) several buyer profiles in the reference period, such situation is evaluated in the most conservative manner. It is not required that all unconcluded contracts were transferred to the newer profile. It is sufficient if at least one of the valid profiles was machine-readable (assuming all data were trasfered here). When evaluating the functionality and quality aspects, only profiles valid at the date of the surveillance were considered. On the other hand, in terms of the completeness aspect, any available data from all profiles past and present was taken into account. Indicator value is bounded below by 0 (negative value can not be assigned even in case of a large number of errors). Machine unreadable profiles contain no tenders and they are thus rated with 0.

The quality of data for small-scale contracts is evaluated rather benevolently. Every such contract with stated at least winning bidder and price is recognized as published. We divide all published small-scale contracts according to their value into following categories: 0 - 250, 250 - 500, 500 - 1000 and 1000+ (in thousands of CZK). A contracting authority receives full score if for every category there are at least as many contracts on the profile as number of contracts in Journal. Each fraction in following formula is bounded from above by 1:

$$ small-scale\:contracts = \Bigg(\frac{contracts\:0-250}{contracts\:in\:Journal} + \frac{contracts\:250-500}{contracts\:in\:Journal} + \frac{contracts\:500-1000}{contracts\:in\:Journal} + \frac{contracts\:1000+}{contracts\:in\:Journal} \Bigg) * \frac{1}{4} $$

The indicator is bounded from below by 0 and thus it can never achieve negative values, even with high number of mistakes. The formula is following: $$z_9 = \frac{1}{2} \frac{number\:of\:linked\:contracts\:from\:profile}{number\:of\:Journal\:contracts} \times \Bigg(1-\frac{number\:of\:mistakes\:on\:profile}{4 \times number\:of\:linked\:contracts\:from\:profile}\Bigg) \\ + \frac{1}{2} small-scale\:contracts$$

Evaluation examples:

• A profile does not have machine-readable interface and thus there is no data about any contracts on this profile: 0 %
• A profile has machine-readable interface, there is sufficient number of small-scale contracts, but only 20 % of Journal contracts, no mistakes: 60 %
• A profile has machine-readable interface, there is sufficient number of small-scale contracts and 100 % of Journal contracts, but 2 mistakes per each tender: 75 %
• A profile has machine-readable interface, there is every single contract from Journal and without any mistake, but no small-scale contract: 50 %

Precise contract publication in compliance with the Decree 168/2016 Coll., On publication of contract notifications pursuant to the Public Procurement Act and on the buyer profile provisions. If even then the profile does not meet the declared requirements, there is typically some issue caused by the profile's provider, who should be notified or possibly even switched. In case of switching it is however still necessary (in compliance with the Ministry of Regional Development's methodology (in Czech)) to transfer all contracts to the new buyer profile. Only in limited number of cases it is possible to leave already concluded contracts on the previous profile, provided it still remains accessible.

A frequent error in terms of completeness aspect is not the actual non-disclosure of a contract on the profile, but rather the incorrect declaration of its Journal registration number, which prevents its identification as the same contract. Such misconduct is considered less serious than the actual non-disclosure.

Evaluation may discriminate contracting authorities that in good faith use profiles, that do not provide a reliable machine-readable interface or such interface do not provide the proper data. Reliability is also assessed in the view of the actual data availability - a number of profiles operate only formally, with a lot of downtime during a single day. If data could not be retrieved on a several attempts, the profile is penalized accordingly.

Said good faith is also understandable in case of instruments whose compliance with the provisions of Public Procurement Act is even officially certified (Ministry of Regional Development's list of certified electronic tools (in Czech)). Nonetheless even these tools struggle with issues that makes it difficult or even prevents the proper publication of information in compliance with the law.

Due to the different implementations of date based search across different profile providers, pertinence of a contract to the reference period may be unclear at times. It is therefore possible that some contracts are evaluated twice. This may negatively impact the final evaluation to a certain degree, if those very contracts are flawed above average.