About this role
The team is seeking a Software Engineering Evaluation Specialist to join their innovative team focused on AI systems. In this role, you will be responsible for designing and evaluating coding tasks that challenge cutting-edge AI coding agents. Your expertise will contribute to the testing and improvement of AI systems in a project-based environment.
Key Responsibilities:
- Design coding tasks that present challenges to AI agents, ensuring they are engaging and effective.
- Create self-contained Docker environments that simulate broken software for AI agents to fix.
- Collaborate with cross-functional teams to refine evaluation criteria and methodologies.
- Analyze AI performance and provide insights for enhancements.
- Document findings and present results to stakeholders.
Required Skills & Qualifications:
- Strong background in software engineering and coding principles.
- Proficiency in programming languages such as Python, Java, or C++.
- Experience with Docker and containerization technologies.
- Familiarity with AI and machine learning concepts.
- Excellent problem-solving skills and attention to detail.
- Strong communication skills in English, both written and verbal.
Experience:
- A minimum of 5-8 years of experience in software engineering or a related field is required.
What we offer:
- Opportunity to work on groundbreaking AI projects with leading tech companies.
- Flexible project-based work arrangements.
- Collaborative and innovative work environment.
Applications are read by our talent team, usually within two working days.
If you look like a fit we will call you, and you will hear from us either way.