How generative AI is shaping the future for technology companies

AI bias can occur when machine learning algorithms inadvertently learn and perpetuate human or systemic prejudices present in the data used for their training. For example, a Microsoft study discovered that AI-generated images could be biased in their representation of gender, race, and age across different occupations and personality traits. As these biases manifest in the generated content, they can perpetuate and amplify pre-existing stereotypes and undermine efforts to promote equality.

Addressing the risks of AI bias requires careful scrutiny of the training data, the development of bias-detection algorithms, and ongoing monitoring to ensure that the AI system is making unbiased decisions. It also underscores the importance of using sufficiently diverse data to train AI systems while avoiding overrepresentation, which is a common problem in large data sets. Selecting the right model for your AI and setting it up correctly lays the foundation for AI systems to operate ethically and in alignment with your organizational goals. Look for models that offer algorithmic transparency.

Tech companies also need to proactively consider the questions or use cases that could be a bias concern and implement strong governance to avoid these issues. For example, you might consider restricting certain topics if they have a high risk of generating biased results.

A hallucination occurs when a generative AI program outputs false information or a response that is not supported by its training data. Since the user does not necessarily see the sources that are used to generate the answer, it can be difficult to distinguish facts from hallucinations, especially because the platform presents them as being factual. Even if sources are cited, the sources themselves may be fake.

Mitigating these risks involves careful monitoring and validation of the AI system's outputs. For example, a company can seek guidance from experts in the field related to the request to verify the output. There’s also the option of designing the platform to include sources, which would allow users to confirm whether the outputs are factual. Asking the AI to include sources—and then validating any sources cited—is a best practice.

Next, tech companies need to train their users on how to properly use the platform and establish clear policies on acceptable and unacceptable use. Training on prompt engineering can significantly reduce the risk of hallucinations by providing users with the knowledge and skills to craft effective prompts that guide AI models towards generating more accurate and contextually appropriate responses.

The risk of sensitive data exposure in generative AI models is a profound concern that warrants strategic attention to secure both proprietary information and user data against inadvertent leaks and potential misuse. Source code stands out as one of the most frequently exposed types of sensitive data. As many generative AI platforms train themselves with data manually input by users, companies should be aware that their employees may have input proprietary data into a generative AI platform. If the data from that application becomes exposed, all user data could be at risk.

The first step to mitigating these risks is understanding the security implications of the platforms in use. For example, if your organization opts to rely on third-party platforms instead of building its own, you need to have a clear understanding of how the data is being used and how long it’s being stored. It’s important to note that not all platforms leverage user data for training purposes.

Safeguarding source code has become an imperative task in the modern digital landscape, necessitating robust data protection measures and constant vigilance to prevent inadvertent data leaks and the potential consequences they may entail.

As with any new technology, tech companies also need to monitor regulatory changes to stay compliant and understand what compliance requirements they may need to address in the future so they can start preparing today.

Innovation Canada is creating a voluntary code of conduct for generative AI systems and is an important resource for tech companies. Companies should also be familiar with Bill C-27 and the changes around how personally identifiable information (PII) is collected, stored, and used. This bill also introduces Canada’s first AI legislation called the Artificial Intelligence and Data Act (AIDA).

To produce sophisticated, human-like outputs, generative AI models undergo training on vast and diverse data sets that often encompass a wide spectrum of sources, including publicly available information, documents, original artwork, and other content types. Some of this data may even come from personal, private, proprietary, or corporate sources, raising critical questions around copyright issues and data theft.

Legal frameworks are yet to fully encompass the realities of AI technology, struggling to define if and how intellectual property rights apply to AI-generated content. The blurred lines between the work of these AI models and original human creativity have sparked discussions about who should own the generated content and how to safeguard against data misuse.

Striking a balance between fostering innovation and protecting the rights of content creators and data owners is a pressing matter in navigating the complex landscape of generative AI. Cases around the world are setting precedents as these technologies continue to evolve and become increasingly mainstream.