The client is a leading U.S.-based AI research and safety company dedicated to building reliable, interpretable, and steerable AI systems.

To enhance the foundational LLM's precision and reliability, our client sought to reduce erroneous outputs or "hallucinations" while expanding the model’s data science and analysis capabilities. As this model forms the backbone of the client’s operations, improving its accuracy and expanding its data handling capabilities were critical goals. The project aimed to evaluate and enhance the model's performance in handling complex data analysis tasks through comprehensive feedback mechanisms.

The client, in collaboration with Turing, initiated a meticulously planned and strategically executed two-phased approach to tackle the challenge. It focused first on creating comprehensive evaluation datasets and then on leveraging reinforcement learning from human feedback (RLHF) to enhance model performance.

• Evaluation dataset creation: Data scientists performed thorough exploratory data analysis to understand the data’s complexities and nuances. This foundational step was crucial in creating up to 20 natural language questions per dataset, organized by complexity (easy, medium, complex) and categorized into data analysis, science, cleaning, and plotting. Each question set underwent a rigorous quality assurance process featuring a dual-development approach. Utilizing Python notebooks within Microsoft Visual Studio Code allowed for a structured and efficient coding environment. Integrating a metadata generator with these notebooks enabled the creation of JSON files that encapsulated essential question metadata and solutions. The key feature of this phase was the “golden answer” methodology, in which only answers that data scientists fully agreed on were used to guarantee the evaluation datasets' high accuracy and reliability.
• RLHF interactions: The team utilized a specialized web interface provided by the client for dynamic interaction with the LLM. This phase was defined by the direct engagement of developers with the models (LLM A & LLM B) as they meticulously evaluated the outputs for accuracy, logical reasoning, and adherence to the golden answers, among other criteria. This evaluation determined the comparative performance of both LLM versions and formed the basis for detailed, constructive feedback to fine-tune the models further. Throughout this extensive interaction process, both offline and live reviews ensured the highest quality of feedback. These reviews, leveraging metadata from each session, fostered a deep understanding of the model’s strengths and areas for enhancement, thereby guiding the continuous evolution of the model’s data analysis capabilities.

The collaborative effort centered on improving the model's data analysis functionalities resulted in the development of over 115 comprehensive evaluation datasets and more than 5,000 RLHF interactions. These achievements included:

• Evaluation datasets: The development and deployment of over 115 comprehensive evaluation datasets, systematically crafted to aid in precise model performance assessments.
• Model accuracy: Substantial enhancements in model accuracy and cognitive capabilities, guided by more than 5,000 extensive RLHF interactions.
• Hallucination reduction: This project was among several initiatives that collectively contributed to a significant decrease in model hallucinations and a 200% uplift in model accuracy. This enhancement has improved the model's ability to perform precise, complex data analysis tasks.