Abstract
Climate change is a pressing issue affecting society, and the Computer Vision community should take steps to limit its environmental impact. This paper analyzes the effect of diminishing returns on CV methods and proposes NoFADE, a novel entropy-based metric to quantify model-dataset-complexity relationships.
Key Contributions
- NoFADE Metric: We introduce an entropy-based metric that quantifies the relationship between model complexity, dataset difficulty, and performance gains.
- Saturation Analysis: We show that some CV tasks are reaching saturation, while others are almost fully saturated.
- Agnostic Comparison Platform: NoFADE allows the CV community to compare models and datasets on a similar basis.
The Problem
As models grow larger and training becomes more compute-intensive, the environmental cost of marginal performance improvements increases dramatically. Understanding diminishing returns is crucial for sustainable AI development.
Methodology
- Analyze performance curves across multiple CV benchmarks
- Compute entropy-based complexity measures
- Quantify the relationship between compute investment and accuracy gains
Key Findings
Many popular benchmarks show signs of saturation where additional computational investment yields minimal improvements. This suggests the community should focus on efficiency and new problem formulations rather than scaling existing approaches.
Impact
This work encourages the CV community to consider environmental impact when designing experiments and to focus resources on problems where significant progress is still achievable.