Talent.com
Mercor
DevOps Engineer AI Model EvaluatorMercor • Rome, Italy
DevOps Engineer AI Model Evaluator

DevOps Engineer AI Model Evaluator

Mercor • Rome, Italy
12 giorni fa
Descrizione dell’offerta di lavoro

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote

Role Responsibilities

  • Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
  • Review model-generated implementations involving cloud platforms Kubernetes CI/CD systems observability and infrastructure automation.
  • Identify bugs edge cases reliability issues and failure modes in model outputs.
  • Compare outputs from multiple frontier models to assess their strengths and weaknesses.
  • Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Qualifications

Must-Have

  • 2 years of professional DevOps SRE or Cloud Engineering experience.
  • Experience with AWS Azure GCP Kubernetes Terraform CI/CD pipelines or observability tooling.
  • Regular use of AI coding agents such as Cursor Claude Code Codex Windsurf Gemini CLI or similar tools.
  • Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Preferred

  • Experience supporting production-scale systems.

Compensation & Legal

  • $400 per accepted task
  • Compensation tied to accepted work.

Application Process (Takes 2030 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information please check:
  • For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.


Employment Type : Full Time
Experience: years
Vacancy: 1

Creare un avviso di lavoro per questa ricerca

DevOps Engineer AI Model Evaluator • Rome, Italy

Offerte simili

Senior AI Engineer – LLMs & Agentic Systems - Freelance

ShakersRoma, lazio, Italy

AI consultancy working on advanced, real-world projects around.AI solutions, not demos or PoCs.DBs, chunking strategies, retrieval & ranking).Developing APIs and services in.OpenAI / Azure OpenAI /... Mostra di più

 • In evidenza

ML Engineer Senior - GenAI Solutions

NTT DATA Europe & LatamRoma, lazio, Italy

As a Senior Machine Learning Engineer at NTT DATA, you will work alongside experienced Data Scientists, Data and ML Engineers on advanced machine learning and Generative AI initiatives.Your Contrib... Mostra di più

 • In evidenza

Lead AI Engineer

IQVIARoma, lazio, Italy

In a world where information is instantly accessible, hospitals remain an exception.Electronic Health Record (EHR) systems are designed to support clinical workflows, but they often fall short when... Mostra di più

 • In evidenza

Senior AI Engineer - Hybrid LLMs & Agentic Systems

Shakersroma, lazio, Italy

A leading AI consultancy in Italy is seeking a Senior AI Engineer to design and build production-grade AI solutions.In this hands-on role, you will work with agentic AI systems and LLMs alongside a... Mostra di più

 • In evidenza

Lead AI Application Engineer (Infrastructure & LLMOps)

TechBiz Global GmbHRome, 62, IT

At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio.We are currently looking for a dedicated.If you're looking for an exciting opportunity to grow in an in... Mostra di più

AI Engineer — Build Enterprise Agents & TS Integrations

Wonderful Ltd.Roma, lazio, Italy

A leading tech company in Rome is seeking a skilled developer to build, test, and deploy AI agents on their platform.The ideal candidate will have 2-6 years of software development experience and a... Mostra di più

 • In evidenza

DevOps Engineer: AI-Driven Pipelines & Observability

Atlas Reply RomaRoma, lazio, Italy

A leading IT services company in Rome is seeking a passionate DevOps Engineer to join their international team.The ideal candidate should have strong knowledge of Python and Bash, experience with D... Mostra di più

 • In evidenza

Senior AI Engineer: NLP, Vision & ML

PwC Italyroma, lazio, Italy

PwC Italy cerca un Senior AI Developer per il suo AI Center of Excellence a Roma.Il candidato sarà coinvolto nello sviluppo di applicazioni innovative basate su intelligenza artificiale, culminando... Mostra di più

 • In evidenza

DevOps Engineer - Kubernetes, OpenShift, CI/CD (Remote 1x/mo)

Experis Italiaroma, lazio, Italy

Experis Italia è alla ricerca di un DevOps Engineer di livello medio per gestire e automatizzare applicazioni containerizzate in ambienti Kubernetes.La posizione richiede esperienza in attività Dev... Mostra di più

 • In evidenza

AI ENGINEER

OKATIM ItaliaRoma, lazio, Italy

In ottica di crescita ed espansione continua, siamo alla ricerca di un.Cliente leader nel settore Aerospazio & Difesa.La risorsa sarà inserita presso il team progettuale del Cliente situato a Roma.... Mostra di più

 • In evidenza

DevOps Engineer

Atlas Reply RomaRoma, lazio, Italy

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.For a new exciting challenge in the.Design, implement and maintain continuous integration and distribu... Mostra di più

 • In evidenza

Remote DevOps Engineer - Terraform, CI/CD & Kubernetes

DPWAY S.r.l.Roma, lazio, Italy

ICT, cerca un DevOps con almeno 2 anni di esperienza.Il candidato ideale deve avere competenze in Terraform, Ansible e CI/CD, oltre a saper gestire container Docker.Questa posizione offre un ambien... Mostra di più

 • In evidenza

Devops Engineer

AyesaRoma, lazio, Italy

Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features.At Ayesa, we grow with you! We don’t wait for the future, we lead it.Through #technology and engineeri... Mostra di più

 • In evidenza

DevOps Engineer - Remote/Hybrid, AWS & CI/CD Pro

AyesaRoma, lazio, Italy

A consulting company is seeking a Mid-Senior level DevOps Engineer to design and develop solutions, manage AWS infrastructure and collaborate with development teams.Candidates should have at least ... Mostra di più

 • In evidenza

Lead AI/ML Engineer for Workflow Tools

ProlificRoma, lazio, Italy

Prolific is seeking experienced AI / Machine Learning Engineer leaders in Roma, Italy to contribute to research projects focused on workflow management tools.Experts will participate in qualitative... Mostra di più

 • In evidenza

Senior AI Engineer — Agentic Systems at Scale (Remote)

Pennylaneroma, lazio, Italy

Pennylane is looking for an experienced AI Engineer to join their Core AI team in Rome, Italy, with responsibilities including building and maintaining agentic systems and improving AI evaluation p... Mostra di più

 • In evidenza

GenAI & LLM Engineer — Remote, Cloud-Native AI Solutions

PROXIMA GROUProma, lazio, Italy

PROXIMA GROUP è alla ricerca di un AI Engineer (GenAI LLM) con almeno 3 anni di esperienza per progettare soluzioni GenAI avanzate.Ti occuperai di pipeline RAG, Vector Database, e orchestrazione d... Mostra di più

 • In evidenza

Senior AI Engineer (Agentic Systems)

Frontiere s.r.l.Roma, lazio, Italy

Senior AI Engineer (Agentic Systems).From feasibility study and analysis of the market and trends, to operations management, through innovation, research and development, we help companies design, ... Mostra di più

 • In evidenza

DevOps Engineer

Optima Solutions S.r.l.roma, lazio, Italy

Per ampliare e consolidare il nostro organico siamo alla ricerca di un bDevOps Engineer /b da inserire all’interno del team tecnico, con l’obiettivo di supportare la gestione, l’evoluzione e l’auto... Mostra di più

 • In evidenza

EO AI Engineer — Agentic Earth Data Pipelines

Pi SchoolRoma, lazio, Italy

Pi School in Rome is seeking an EO AI Engineer to enhance EVE, an open-source AI platform for Earth Observation.You will work with satellite data and develop AI-powered agents to integrate EO data ... Mostra di più