UAE's TII Launches AI Challenge On Crowdsourced Labeling

The UAE’s Technology Innovation Institute (TII), part of Abu Dhabi’s Advanced Technology Research Council (ATRC), has launched the CrowdLabel Challenge with the aim to develop methods for passive human labeling – a technique of collecting labeled data without requiring explicit labeling efforts from individuals – to support the training of large language models (LLMs).  

LLMs use a technique called reinforcement learning from human feedback (RLHF) to align intelligent agents with human preferences. However, using human labelers can be costly. TII’s challenge therefore seeks to identify a passive approach to human labeling, where the act of labeling data samples is integrated into other tasks, allowing humans to label data, without being aware of it. Besides lowering costs, TII believes that such passive alignment methods can be applied to a variety of everyday activities, such as bot detection systems.  

As part of the challenge, TII has established a database of alignment samples to help LLM researchers gather scalable and reliable feedback for their models. The challenge is looking for solutions that can handle multiple languages (at least two), and can support multimodal content as well as a range of label types on various topics.  

Entries for the challenge, which is open to individuals, teams, startups, research institutes, and university students worldwide, will be judged based on the approach, cost-effectiveness, and the quantity and quality of labels produced during an experimental phase. The total prize pool for the competition is US$50,000, with $30,000 set to be awarded to the first-place winner. 

Submissions for the CrowdLabel Challenge will be accepted until February 28, 2025; click here to learn more about the initiative.