1
PythonReactVueAWS
1
Role overview
Job description
We’re looking for an AI Engineer to help us design, evaluate, and scale our next generation of LLM-powered agents. This role is deeply technical, but also product and customer-oriented: you’ll build datasets, run evaluations, improve model performance, and ensure our agents deliver real value in real workflows.
As an AI Engineer, you’ll work across Engineering, Product, and Customer teams to continuously improve our agent quality, reliability, and speed. You won’t just build agents — you’ll build the systems, metrics, and feedback loops that make those agents better over time.
Responsibilities
Build, curate, and maintain agentic workflows along with datasets for training and evaluation. Run systematic LLM evaluations, track regressions, and ensure models meet quality bars. Define and implement LLM performance metrics(e.g., correctness, latency, hallucination control, safety). Work closely with Product and Customer-facing teams, experiment with prompting techniques, fine-tuning datasets, retrieval strategies, and model configurations and ensure agent behaviour aligns with user expectations. Develop internal tooling to speed up evaluation, annotation, and iteration cycles. Build automated pipelines for regression checks and model monitoring. Create mechanisms that turn real user interactions into actionable model improvements. Ensure agents behave consistently across large-scale production scenarios. Debug complex system behaviours spanning prompts, tools, APIs, and model responses. To understand real-world use cases and failure modes. Translate customer insights into model needs, data requirements, and product improvements. Tune latency, turn-taking, and conversational naturalness for voice AI systems. Write clean, reliable code in Python or JS for model pipelines, tools, and integrations. Think in systems: from data ingestion to model outputs to user-facing behaviour.
Success Looks Like
Strong evaluation coverage with clear, actionable metrics. Faster iteration cycles due to improved internal tools and workflows. Measurable improvement in agent accuracy, consistency, and safety. Reliable agent performance in production — fewer escalations, fewer regressions. Clear alignment between customer needs and agent capabilities. Smooth collaboration across engineering, product, and customer teams.
What We’re Looking For
2–5 years experience in software engineering, with strong fundamentals in building reliable, scalable systems.1–2 years of experience working on AI-backed products(LLM engineering, evaluations, prompting, or agent development). Hands-on experience with LLMs, prompt engineering, evaluation frameworks, or dataset creation. Strong understanding of how LLMs work: prompting, fine-tuning, evaluation, guardrails, and system design. Solid software engineering skills and comfort with APIs and cloud environments. Experience building internal tools or automation pipelines. Ability to think from the customer backward: empathetic, detail-oriented, and outcome-driven. Excellent communicator who collaborates well across cross-functional teams. Fast learner who loves turning cutting-edge research into practical, dependable systems. Bonus: experience with voice AI, speech technologies, or real-time agent systems.
Why This Role Matters
In healthcare, accuracy and trust are everything. Patients, providers, and payers rely on our AI agents to handle critical interactions. Deployment Engineers make this possible by turning AI from a raw model into something practical, reliable, and human-friendly.
Why 100ms.ai
You'll be part of a small team at a fast-growing engineering-first startup. You'll work with engineers across the globe with experience in video at places like Facebook and Hotstar. You can grow as an individual contributor or as a team leader - freedom to set your own goals.
More like this
About this listing
- Source: 100Ms's official careers site · still listed there on 7 Nov 2026
- Posted 20 Nov 2025
Jobiba does not charge job seekers. Never pay to apply or share OTPs, card or bank details.
Role market pulse
How AI Engineer demand looks in India
Live jobs
15
Posted 30d
11
30d movement
+175%
Remote share
20%
Salary listed
26.7%
Salary trend 1Y
Not enough history
What they are looking for
Skills & requirements
Qualification
We’re looking for an AI Engineer to help us design, evaluate, and scale our next generation of LLM-powered agents; Run systematic LLM evaluations, track regressions, and ensure models meet quality bars; Define and implement LLM performance metrics(e.g., correctness, latency, hallucination control, safety); 1–2 years of experience working on AI-backed products(LLM engineering, evaluations, prompting, or agent development)
Department · AI Team
Hiring company
100Ms
View the company profile, current openings and hiring details.
Related opportunities
Similar active roles
Relevant, currently active roles for this market.
1
Bengaluru, KA, India
₹50L – ₹1.5Cr/yr
PythonReactAWSAzure
S
Bengaluru, KA, India· Full-time· Hybrid
PythonJavaAzureAI
B
Bengaluru, KARNATAKA, India· Hybrid
TypeScriptNode.jsAzureKubernetes
BA
Bengaluru, KARNATAKA, India· Full-time
AILogistics
C
Bengaluru, India
PythonSQLArtificial IntelligenceAI
🔔 Get job alerts
New AI Engineer jobs in Bengaluru, KA, India, straight to your inbox.
No spam · Unsubscribe anytime