ChatGPT - legal challenges, legal opportunities
Locations
After trying chatting with ChatGPT, most users report being extremely impressed, some even anticipating a fundamental change in the world as we know it. However, fears are also expressed by some users – including knowledge workers becoming obsolete and potential dangers to democracy. While it is too early to assess all possible effects of ChatGPT, including those from a legal point of view, certain matters immediately raise specific questions.
Introduction to ChatGPT
In order to assess the legal implications that ChatGPT entails, we need to understand what ChatGPT is and how it works. And who should answer our questions? Of course: We asked ChatGPT to summarise what it is. This is ChatGPT's response:
"ChatGPT is a chatbot that uses a large language model trained by OpenAI. It is a variant of GPT-3 (Generative Pretrained Transformer 3), which is a state-of-the-art natural language processing model. ChatGPT has been specifically trained to respond to human input in a conversational way, and it can generate human-like text based on the context of the conversation. This allows ChatGPT to engage in natural-sounding conversations with users on a variety of topics. ChatGPT can be used in a variety of applications, including customer service, chatbots, and virtual assistants"
When asked how ChatGPT functions, ChatGPT replied:
"ChatGPT works by using a large language model trained by OpenAI. This model has been specifically trained to generate human-like text based on the context of a conversation. When a user enters a message, ChatGPT processes the input and generates a response based on the words and phrases in the message. The response is generated in real-time, and ChatGPT continues to update its response as the conversation progresses. This allows ChatGPT to engage in natural-sounding conversations with users on a variety of topics."
Many readers of this text will have already tried ChatGPT and know how impressive the technology is. Even to complex questions, ChatGPT finds answers that could come from someone with expert knowledge.
But does ChatGPT copy texts from the Internet? The AI itself explicitly denies that.
"ChatGPT does not simply copy content from the internet. It is a large language model that has been trained using a vast amount of text data, including books, articles, and other written material. This allows ChatGPT to generate responses based on the context of a conversation in a way that sounds natural and human-like. While ChatGPT may use information from the internet to generate its responses, it does not simply copy and paste content from websites"
Possible use-cases of ChatGPT in the commercial sector
We asked ChatGPT how companies can use it. ChatGPT came up with an array of possibilities:
- "Customer service: ChatGPT can be used to provide automated customer support, answering common questions and providing information to customers.
- Virtual assistants: ChatGPT can be used as a virtual assistant, helping users to manage their schedules, make reservations, and perform other tasks.
- Chatbots: ChatGPT can be used to create chatbots for websites, messaging apps, and other platforms.
- Content generation: ChatGPT can be used to generate content for websites, social media posts, and other written materials.
Given the buzz ChatGPT generates on social media, these possibilities are a small sample of ChatGPT's potential use cases. Early users are already conjuring up the end of all sorts of professions that process or present knowledge in an orderly fashion, such as journalists or nonfiction writers. But even lawyers could soon be superfluous, some say - not a promising outlook for the authors of this article, but we accept the challenge sportingly.
But can the content really be used?
However, the question arises to what extent the commercial use of ChatGPT - which so far can be used free of charge by anyone - is allowed and has to be remunerated - with respect to both OpenAI and third parties, with whose contents ChatGPT was fed and which thus indirectly contributed to ChatGPT's results. We asked ChatGPT, if it is legal to use the content produced by ChatGPT in commerical use cases. Its response sounds as follows:
"It is generally legal to use the content produced by ChatGPT in commercial use cases, provided that you have the appropriate permissions and licenses. As a large language model trained by OpenAI, ChatGPT generates text based on the context of a conversation, and the specific responses it produces will vary depending on the input it receives. If you want to use the content generated by ChatGPT for commercial purposes, you may need to obtain a license from OpenAI or the relevant rights holders, depending on the specific circumstances of your use case. It is always important to ensure that you have the necessary permissions and licenses before using any content for commercial purposes."
From a lawyer's point of view, this answer is not helpful. On the contrary: It remains questionable and shifts the responsibility to the user whether he needs a license from OpenAI for commercial use for the content he generates with ChatGPT. And beyond that? What if ChatGPT is asked to briefly summarize complex content that is copyrighted, such as condensing a book of 300 pages? Does the user need paid permission from the author or publisher because that is essentially a service that service providers such as Blinkist offer? This is just one example of the many copyright issues that arise.
With this, ChatGPT does impose many relevant questions, while being unable to answer them conclusively. It remains to be seen how OpenAI and third parties affected will react to potential commercial use of ChatGPT in the future.
Potential legal issues?
ChatGPT itself appears to be unconcerned about possible legal issues. When we asked, whether any of these existed, ChatGPT responded:
"I am not aware of any legal issues specifically concerning ChatGPT. As a large language model trained by OpenAI, ChatGPT generates responses to user input in a way that is similar to how a human would respond. The responses it produces will vary depending on the context of the conversation, and it does not simply copy content from the internet. As long as you use ChatGPT in accordance with the terms and conditions set by OpenAI, and obtain the appropriate permissions and licenses if necessary, there should not be any legal issues. It is always important to consult with a legal professional if you have specific questions or concerns about the use of ChatGPT or other AI technologies."
The last highlighted sentence emphasizes that the creators of OpenAI are well aware that there may still be many legal problems with ChatGPT - and refer to legal experts for this. Perhaps this is how a new industry of lawyers will emerge - AI understanders or AI analysts who will evaluate a Ki's response or work for its legal harmlessness for use. But then, unfortunately, the question of whether to hire a lawyer arises again. Everyone has to decide for themselves - and the authors of this article are biased in their analysis here.
ChatGPT in the legal industry
As for the legal industry, while some users tweet that ChatGPT will take legal advice from human lawyers obsolete or at least significantly reduce their time commitment to specific tasks, this is still far from reality.
ChatGPT, in essence, generates its answers by putting pieces of text together based on a statistical model trained on billions of text samples from across the Internet. While this can drastically reduce time spent on legal research and collecting precedent when the task ChatGPTis given does not merely entail gathering and summarising information but requires critical thinking and elaborate application of legal principles, it is prone to provide incorrect answers.
For example, we asked ChatGPT what an "Anfechtungsklage" (legal challenge) is in a German administrative court. While the primary explanation was correct, ChatGPT claimed that the court set the deadline for filing a lawsuit. This is grossly incorrect. The time limit for filing a case one month after receipt of a negative administrative act is regulated by law and cannot be extended even by a court. Anyone who relied on this information would have run into a dangerous misconception. Which again raises the next question. Who would have assumed liability for this?
ChatGPT is not a knowledge database. ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. OpenAI also points out that ChatGPT cannot provide accurate information on topics and changes after its knowledge cutoff in 2021. However, currently there are no terms of use, and liability issues will certainly need to be addressed before ChatGPT is put into productive use.
Incidentally, similar issues arise when ChatGPT is used for programming purposes. Hence, Stack Overflow, a software forum, has already banned replies generated by ChatGPT due to their susceptibility to errors.
ChatGPT and the public sector
In addition to possible commercial use, the question of ChatGPT being utilized by the public sector. Queried about this, ChatGPT came up with an answer almost identical to the one given regarding commercial use and does not contain theoretically conceivable areas of application such as statute drafting, bid selection in procurement procedures and judicial and administrative decision making. The extent to which such tasks can be delegated to AIs, the level of human control over them, and how the use of AI will affect legal protections raises countless legal questions.
- Are authorities allowed to use AI at all? This raises profound legal questions, even before the background that OpenAI controls AI.
- Can the authorities trust the information provided by OpenAI?
- Does the user have to be differentiated according to how legally sensitive the area of use is? For example, where the state decides life and death or the individual's freedom, even such a highly developed AI as ChatGPT should not be used.
And what about the protection of personal data, especially compliance with GDPR? ChatGPT itself seems to be unable to elaborate on this, answering our query about GDPR compliance as follows:
"I cannot say for certain whether ChatGPT is GDPR compliant, as I am not aware of its specific design or implementation. The General Data Protection Regulation (GDPR) is a European Union (EU) law that sets out rules for the collection, use, and storage of personal data. It applies to organizations that process personal data of individuals who are located in the EU, regardless of where the organization is based. Whether ChatGPT is GDPR compliant will depend on how it is designed and used, and whether it collects, processes, or stores any personal data of individuals who are located in the EU. If you have specific concerns about the GDPR compliance of ChatGPT, I would recommend consulting with a legal professional."
ChatGPT actually comes up with all the right variables necessary to assess whether GDPR standard are met but is unable to assess them accordingly.
Political use
And what about the big picture – can this AI endanger our democracies? ChatGPT can easily write pieced of political advice, praise or critique; it could be used by lobbyists, activists but also opens the door to potential abuse by extremists. Its ability to produce legit-sounding convincing content might also facilitate the production and spread of convincing fake news and hate speech with potential serious effects on democracy – extremely fast and accurate. What if the tool gives you the idea of speaking to an actual person via a chat, making it sound reasonable to have extreme views. Certainly this is now what OpenAI aims for or supports, but can it actually prevent this from happening once it gives out licenses all around the world to use the AI technology and alter it?
Additional dangers?
ChatGPT has certain safeguards that prevent it from outputting some particularly offensive and harmful content. These, however, are not free from errors and can be bypassed which has, for example, resulted in users tricking ChatGPT into providing them with a guide to shoplifting or building a Molotov cocktail. There is also the obvious risk that deceptively legitimate texts created by ChatGPT could be used for scams – like phishing emails - , blackmail and other fraudulent activity. ChatGPT Naturally, this raises liability questions.
Similar question arise when ChatGPT-created texts violate personal rights, e.g. by producing insulting or misleading content on a person or are used to spread misinformation that leads to infringes a company's reputation. There is also the risk of discrimination based on race, gender or physical traits by ChatGPT if it is instructed accordingly.
Conclusion
ChatGPT is a fantastic technology and a prime example of how quickly AI can reach the entire world. And yet, there is still a multitude of legal issues and legal questions that could still stand in the way of widespread use. Against this rapid development, it seems sensible for the European Union to address such issues precisely with the planned AI Act. ChatGPT will also be measured against the rules of this legal regulation once it comes into force. Whether the dangers and open legal questions will become fewer remains to be seen.
Stephan Zimprich is a technology lawyer and a partner in Fieldfisher's Hamburg office. He regularly publishes and lectures about AI-related legal topics.
​Dennis Hillemann is a specialist in administrative law and a partner in our Hamburg office. He has recently published and lectured on the use of AI in the public sector. In addition, he advises companies and public authorities on digitalization issues.