AT&T Boosts Customer Service AI Using NVIDIA NeMo Technology

AT&T has partnered with NVIDIA to revolutionize its customer service operations using AI-powered agents built on NVIDIA NeMo and NIM microservices. The telecommunications giant is addressing key challenges in AI deployment, including model accuracy, computational costs, and real-time data access.

Key Developments

• Data Flywheel Approach: AT&T implemented a continuous feedback system using NVIDIA NeMo to maintain agent accuracy as documents update weekly
• Performance Gains: Achieved 40% improvement in response accuracy and 84% reduction in call center analytics costs
• Optimized Models: Fine-tuned Mistral 7B emerged as the most efficient model after testing multiple options

Technical Implementation

AT&T's solution involves:

1. NeMo Curator for data cleansing and filtering
2. NeMo Customizer for model fine-tuning
3. NeMo Evaluator for performance measurement (Rouge, BERT F1 scores)
4. NeMo Retriever for real-time data access
5. NIM microservices for secure, optimized deployment

"The successful fine-tuning story of this use case and others like it was enough evidence for us to pursue building out an entire fine-tuning platform" - Kostikey Moustakas, Director of Data Science at AT&T

Future Plans

AT&T is developing:

• A centralized feedback system combining human and AI evaluation
• Automated handling of complex AI interactions with Arize AI
• A platform for scalable AI optimization across the enterprise

This initiative represents a significant step in AT&T's strategy to deploy dozens of AI use cases, with hundreds more in development.