NVIDIA Launches General Availability of NeMo Microservices for AI Development

Multinational technology company NVIDIA has announced the general availability of its NeMo microservices, designed to help enterprises build AI teammates that scale employee productivity using data flywheels. These microservices provide an end-to-end platform for creating AI agents that leverage human and AI feedback, informed by inference and business data.

Key Features of NeMo Microservices

• NeMo Customizer: Accelerates LLM fine-tuning, delivering up to 1.8x higher training throughput using techniques like supervised fine-tuning and low-rank adaptation.
• NeMo Evaluator: Simplifies AI model evaluation with just five API calls, enabling custom and industry benchmarks.
• NeMo Guardrails: Enhances compliance protection by 1.4x with minimal latency (half a second), helping IT teams implement robust safety measures.

Partner Success Stories

• AT&T achieved a 40% accuracy boost for its AI agents using NeMo microservices.
• Cisco partnered with Galileo to develop a coding assistant with 40% fewer tool selection errors and 10x faster response times.
• Nasdaq integrated NeMo Retriever microservices into its GenAI Platform, improving search accuracy by 30% and reducing costs.

Ecosystem and Integration

NeMo microservices support a wide range of open models, including:

• Llama
• Microsoft Phi
• Google Gemma
• Mistral
• Llama Nemotron Ultra

Meta is leveraging NeMo microservices through connectors for Meta Llamastack, enabling continuous model optimization. Other software providers like Cloudera, Datadog, and DataRobot have also integrated NeMo microservices into their platforms.

Future Outlook

NVIDIA remains committed to advancing agentic AI for enterprises and regulated industries. The company recently partnered with Google Cloud to develop secure, on-prem AI solutions using Gemini models. Learn more about the partnership here.

"Our view is that AI teammates will help over a billion knowledge workers across industries," said Joey Conway, Senior Director of Generative AI Software for Enterprise. "NeMo microservices are the easiest way to scale AI workforce productivity."

For more details, visit NVIDIA's official site.