Balancing Risk and Opportunity: Implementing LLMs with a Security-First Approach

With the promising expansion of large language models (LLMs) across industries, organizations are actively assessing how they can leverage generative artificial intelligence (GenAI) and LLMs to generate business value. From enhancing scientific research and legal analysis to virtual assistance and content generation, LLMs are redefining possibilities of how technology can transform the way businesses operate. A McKinsey research states that “Generative AI’s impact on productivity could add trillions of dollars in value to the global economy.” However, this traction comes with a steep incline in potential security and privacy risks that demand a careful strategy to manage. 

While every business must be vigilant about security risks and data privacy, these issues are particularly important to sectors like banking, credit evaluation, insurance, healthcare risk profiling, and clinical study interpretation, where the impact of such errors can be especially far-reaching. In the OWASP Top 10 for LLM report, over 100 experts, including security specialists, AI researchers, and developers, have identified the top 10 security risks that businesses must consider when building LLM applications. These risks include prompt injections, data leakage, and insecure plugins. 

In this blog, we explore the key challenges and risks associated with implementing LLMs. We also discuss strategies to mitigate these risks and ensure safer LLM deployment. 

Let’s dive in!

LLMs combine the power of AI and deep learning algorithms, powered by substantial datasets to simulate humanlike text generation. These models have the unique ability to comprehend and respond to natural language prompts with a text-based output of a quality once solely associated with human intellect.

These models go beyond mere text generation; they execute complex tasks like sentiment analysis, document summarization, AI-augmented software development, and language translation. To estimate the power of GenAI and LLMs, consider Anthropic's enhancement of Claude in May 2023. Anthropic increased the model's processing limit from 9K to 100K tokens, allowing it to process around 75,000 words per minute.

There are two types of LLMs: open-source LLMs and proprietary LLMs.

Proprietary LLMs are built and owned by businesses. To use these, you need to buy a license from the company that specifies the permitted uses of the LLM, typically restricting its redistribution or modification. Examples of proprietary LLMs include PaLM by Google, GPT by OpenAI,  and Megatron-Turing NLG by Microsoft and NVIDIA.

Open-source LLMs, on the other hand, are a shared resource. They're free for anyone to use, modify, and distribute as they like. This openness can encourage a lot of creativity and collaboration. Examples of open-source LLMs include CodeGen by Salesforce and LLama 2 by Meta AI.

Whether you're tapping into the streamlined service of a proprietary model or harnessing the collaborative potential of an open-source alternative, cautious engagement is paramount when working with confidential business information. In the next section, we explore significant business use cases where LLMs are making their mark and touch upon the associated concerns that need to be diligently managed.

The transformative impact of LLMs is notable in significant business use cases, including:

• Scientific research: LLMs facilitate data analysis and hypothesis generation. However, risks arise from potential biases in the models that could lead to flawed research outcomes.
• Legal document review and analysis: LLMs expedite document review, but the possibility of missing critical or sensitive information due to oversights in training data poses a legal risk.
• Copilot assistants: LLMs assist developers in coding, but the risk of generating insecure code snippets remains a critical concern for software security.
• Content creation: The creation of marketing material can be enhanced by LLMs, though the authenticity and accuracy could be compromised and lead to misinformation.
• Virtual tutoring: Personalized educational content is an LLM feature, but the potential exposure of student data is a risk that must be managed.\
• AI-driven chatbots: LLMs improve customer service efficiency but may inadvertently leak personal information or provide incorrect or biased information.

LLM deployment can encounter several challenges that hinder their performance and integrity, including:

• Information bias or misinformation: As powerful as they are for crafting content, LLMs carry a hidden challenge: their susceptibility to ingrained biases. These biases, if left unchecked, can distort the transformative potential of LLMs, making the push for fair and equitable AI development an urgent priority.
  
  Without careful evaluation, AI-generated content could inadvertently propagate misinformation or reinforce societal biases—outcomes that would undermine the very benefits these technologies promise. Experts like MIT CSAIL postdoc Hongyin Luo highlight that current language models suffer from issues with fairness, computational resources, and privacy.
• Environmental impact and energy consumption: LLMs are not only expensive to operate but also carry a substantial carbon footprint. According to Sajjad Moazeni, an assistant professor of electrical and computer engineering at the University of Washington, executing numerous LLM queries may consume around 1 gigawatt-hour. To put this into perspective, that’s equivalent to the daily energy usage of approximately 33,000 U.S. households.
• Cost: The financial barriers to developing LLMs are steep. These models require the power of thousands of GPUs to process the extensive data they learn from. The investment in the necessary GPUs alone can surge into the millions of dollars. For perspective, training OpenAI's landmark GPT-3 would cost over $4.6M using a Tesla V100 cloud instance.
  
  But that’s only a fraction of the total expense. Multiple training iterations are essential during the development and refinement stages, which significantly drive up the overall cost. Speaking at an MIT event, OpenAI’s co-founder Sam Altman suggested that training such foundational models could exceed expenses of $50 to $100 million—numbers that he hinted are climbing higher.
  
  In addition, Sam Mugel, CTO of Multiverse, forecasts that costs for training future generations of LLMs could soon cross the billion-dollar threshold, a figure that’s indicative of the vast resources these models consume.

Implementing LLMs involves intrinsic risks that must be preemptively identified and managed, including:

• Prompt injection:  This is an exploitative tactic where a user intentionally inputs a malicious or deceptive prompt to manipulate the model to produce an undesired outcome or response. Systems with advanced features like API integration or code execution are especially at risk to this tactic. These specially designed prompts can lead to several potential risks, including bypassing content filters, revealing sensitive information, and executing unintended actions.
• Oversharing sensitive data: One of the significant risks in implementing LLMs like ChatGPT lies in the inadvertent oversharing of sensitive information. As part of their learning process, LLMs often utilize the data provided in user interactions, which can include confidential details accidentally shared during the conversation. 
  
  The implications of such data exposure have led to reactive measures by major corporations. A notable instance was when Samsung, following a privacy mishap, decided to restrict the use of ChatGPT across its systems to protect against the leak of sensitive business intelligence. This stance is echoed by industry giants like Amazon, JP Morgan Chase, and Verizon, who all limit the usage of AI tools that could compromise corporate data security.
• Security concerns: A new report launched by Rezilion, an automated software-supply-chain security platform, reveals that the world’s most popular GenAI projects present a high-security risk to organizations. Its research team analyzed the top 50 GenAI projects on GitHub and used the Open Source Security Foundations’ Scorecard to gauge the overall security health of the open-source LLM landscape. The Scorecard is an automated tool that evaluates critical security measures, known as "checks," which are essential for software safety, assigning a score between 0 and 10 for each criterion. A score of 10 indicates the project is highly concerned about security, whereas a lower score suggests that the project’s security is predominantly user-dependent. 
  
  As per the report, none of the projects achieved a score above 6.1 out of 10, settling at an average of 4.6 which flags a significant security concern. Even high-profile projects like Auto-GPT, based on GPT-4 and with a following of over 138,000 stars, barely managed a score of 3.7, underscoring the potential security risks these projects present.
  
  Alongside concerns about the leakage of personally identifiable information, there's apprehension that sophisticated LLMs could be repurposed by cybercriminals for nefarious activities like phishing or spam campaigns. Hackers may manipulate LLMs to offer sophisticated technical advice on executing cyber attacks, potentially enhancing hackers’ capabilities.

Anticipating and managing the risks associated with LLMs demands a multi-layered approach focused on prevention, preparedness, and responsiveness.

• Data security and access controls: Employing encryption, data sanitization, and anonymization protocols are critical in safeguarding sensitive information. Rigorous access controls and constant monitoring can mitigate unauthorized data access and misuse. Isolation of LLM training environments can also prevent unauthorized data exposure during model development.
• Compliance with data protection regulations: Enterprises must ensure that their LLM usage adheres to data privacy laws such as the General Data Protection Regulation (GDPR) or the Health Insurance Portability and Accountability Act (HIPAA). Noncompliance could lead to significant legal and monetary penalties.
• Incident response plans: To effectively manage risks while deploying LLMs, businesses must implement a thorough incident response plan. This plan involves four critical stages: 
  a. Preparation, where a business identifies its digital assets and potential threats, and assembles a dedicated response team.
  b. Detection and analysis, which calls for vigilant monitoring of systems to quickly spot and evaluate any security incidents.
  c. Containment, eradication, and recovery, where the team acts to limit damage, remove the threat, and restore systems with as little disruption as possible.
  d. Post-incident phase, which focuses on analyzing the event, refining the response plan, and maintaining transparent communication with all stakeholders involved.
• Comprehensive AI ethics training: By investing in education that underscores the ethical implications and boundaries of AI technology, businesses can empower their workforce to wield AI responsibly. This awareness is integral to preventing misuse and recognizing the limitations of AI applications.
• Environmental sustainability: To mitigate the environmental cost of LLMs, the industry is actively pursuing sustainability in AI development. Innovations in energy-efficient model architectures, such as sparse networks, are under exploration to maintain AI performance with less energy consumption. Simultaneously, there is a shift toward greener infrastructure, with companies investing in data centers that run on renewable energy sources like wind and solar power.
  
  To complement these efforts, techniques like knowledge distillation are being employed to create streamlined models that retain essential capabilities but with lower energy requirements, and transfer learning utilizes the learnings from pre-existing models to reduce the need for intensive retraining. Additionally, model pruning and compression are cutting down on computationally expensive redundancies within models, demonstrating that eco-friendly approaches can coexist with technological advancement in AI.
• Responsible AI governance structure: A multidisciplinary governance team of data scientists, ethicists, and legal experts can collectively navigate the complex ethical terrain and legal stipulations related to AI. Such a team ensures that training datasets are carefully chosen to reflect diversity and fairness while also establishing and upholding strong governance protocols. Continuous vigilance in the form of monitoring and auditing AI outcomes is crucial for honoring ethical commitments.
• Using guardrails: Implementing technical measures such as guardrails can help maintain the integrity of LLM outputs. These mechanisms enforce constraints that prevent the generation of biased, offensive, or inaccurate responses. One common method involves the use of blacklists and whitelists. A blacklist contains terms that the LLM is restricted from using, whereas a whitelist promotes the use of preferred terms, ensuring outputs align with expected standards.
  
  Another effective strategy is the deployment of filtering systems that scrutinize and eliminate any objectionable material produced by the LLM, such as hate speech or other forms of harmful content. These proactive steps form a protective layer that guides LLMs to function responsibly and align with organizational values and norms.

As we continue to harvest the capabilities of LLMs, the focus must shift from only utilization to conscientious deployment. A cross-disciplinary partnership between data scientists, cybersecurity experts, and regulatory authorities is key to crafting robust frameworks that govern the ethical use of LLMs and ensure that privacy, safety, and legalities are not compromised. By adopting comprehensive data governance, enforcing rigorous model validation, and advocating for transparent operations and continual monitoring, companies can navigate the complex landscape of LLM implementation. 

Companies can also partner with expert tech services companies, like Turing, to unleash the true potential of LLMs. Turing has helped the world’s leading LLM companies with code generation and reasoning training solutions, generating custom proprietary data, and fine-tuning, adapting, and enhancing LLM training capabilities. 

Talk to an expert today and scale with Turing’s custom LLM training and development services.