Special Issue on Performance Evaluation in Computer Vision

In this special issue we sought all types of contributions broadly relating to performance evaluation in computer vision, including:

Manuscripts introducing new performance evaluations or benchmarks, in particular in areas where quantitative evaluation is challenging or subjective

Manuscripts surveying, evaluating, or comparing existing benchmarks or datasets

Manuscripts addressing pros and cons of performance evaluations and datasets, as well as recommendations for best practices

We received 38 initial submissions for this special issue. Of these, 30 were considered to be within the scope of the special issue and underwent a rigorous peer review process. Each paper received at least 3 reviews, for a total of 148 reviews by experts in the field—we greatly thank all of them for their effort! In the end, we accepted 18 papers for publication, all of which went through at least one round of revisions.

The accepted papers span the following topics: datasets, benchmarks and evaluations, robustness, novel metrics, and bias in datasets.