Bloomberg runs on data. Our products are fueled by powerful information. We combine data and context to paint the whole picture for our clients, around the clock – from around the world. In Data, we are responsible for delivering this data, news and analytics through innovative technology - quickly and accurately. We apply problem-solving skills to identify innovative workflow efficiencies, and we implement technology solutions to enhance our systems, products and processes - all while providing customer support to our clients.

Our Team:

Data AI contributes to the building of Bloomberg’s AI-enhanced products at scale by curating model training data and enhancing how our internal processes use AI. By investing in AI at a strategic level, we expand our practice of engaging with AI to one that is embedded across Data. We encourage our internal processes to take advantage of new AI technologies and strengthen Data’s role in providing robust domain expertise and influential data artifacts to Bloomberg’s products. This way, our clients will continue to have high quality data and access to new types of datasets.

The Role:

The AI Safety team supports the governance of Bloomberg’s AI-enhanced products at scale by curating model training data and improving the safety service behind our internal processes that use AI. As a member of the Data AI Safety team, you will be instrumental in coordinating and detailing annotation projects with human annotators, product managers and machine learning engineers. You’ll also have the opportunity to solve innovative problems in machine learning, particularly regarding LLM's. The ideal candidate will have experience working with risk taxonomies, applying English language knowledge to natural language processing, red-teaming project management and coordination, and people management experience.

We’ll Trust You To:

• Plan and implement red-teaming events for Bloomberg’s AI product suite, and develop red-teaming guidelines, strategies, and read outs.

• Relay and interpret project guidelines and instructions for red teamers and annotators

• Train in-house annotators so that they are able to interpret project guidelines using a combination of evaluation metrics to assess their performance to ensure we are producing high quality data that is fit for purpose.

• Serve as an interface for feedback and questions from red teamers and annotators back to data scientists.

• Prioritization and scheduling of projects to align with team goals and timelines.

• Work with data scientists and red-teaming engineers to distill findings into recommendations for product engineering teams.

• Supervise and manage individual annotator performance metrics (accuracy, efficiency, etc.) while offering guidance for improvement.

You’ll Need to Have:

• 4+ years of professional work experience related to linguistic annotation tasks

• Strong understanding of English language (linguistics, syntax, grammar, cognitive science) and its applications to Natural Language Processing.

• Experience working on crowd-based annotations projects.

• Experience developing, implementing, and communicating a sophisticated ontology or taxonomy system.

• Project management expertise: Ability to collaborate with team members to prioritize competing projects, set and maintain a schedule for deadlines and project completions, connect with all levels of engineers and in-house annotators.

• Work with highly-sensitive content with exposure to offensive and sensitive content.

We’d Love to See:

• Experience coordinating red teaming events

• Experience with Risk Taxonomies

• Experience working with annotation schemas, edge cases, guideline development and maintenance, and semantic analysis

• Experience redefining workflows into a more timely and efficient process

• Strong desire to structure and systemize processes, and motivation to push both existing and new workflows in that direction

• Experience using native language skills to collect various forms of linguistic utterances with high accuracy

• Experience working with human in the loop workflows