Synthetic Media: A Double-Edged Sword in the Digital Era

The rise of synthetic media, its implications, and the pursuit of authenticity.

Donald Trump and Vladimir Putin holding hands while the press looks on, generated by Midjourney v5.1

Unless you have been living in a cave for the past 6 months, you have undoubtedly seen no less than 10 posts per day on your social media feeds about ChatGPT, GPT-4, open source large language model (LLM) and open-source chat-bot alternatives like LLaMA and OpenAssistant. This has been accompanied by extensive public discourse and polarization amongst industry leaders on whether there should be a pause on training large language models more powerful than GPT-4 or how there is now an extensional threat to humanity due to the power of generative AI.

I personally, am not at one extreme side of the argument or the other as I recognize what AI is currently capable of as well as what humans are capable of. As Stephen Hawking famously said before his passing:

“The rise of powerful AI will either be the best or the worst thing ever to happen to humanity. We do not yet know which. The research done by this centre is crucial to the future of our civilisation and of our species.”

-All technology has the potential for misuse and AI is no exception.-

A Quick Note on the State-of-the-Art in LLMs

The LLM-ChatBot debate has been raging on and has garnered the most attention in public discourse and in the media. Language is powerful, and when we see a system producing coherent responses to our questions or instructions, we assume there must be some intelligence behind it (intelligence is a loaded term, but here let’s refer to the generally accepted psychology definition that covers cognitive processes such as reasoning, problem-solving, generalization, cognitive mapping, abstract thinking, comprehension and introspection.) However LLMs, which are primarily built on the transformer architecture are auto-regressive (sequential) models heavily loaded with knowledge scraped from the public Internet, they function by predicting the next most likely word in a sequence in order to form coherent and contextually relevant responses. They do this by processing huge amounts of unstructured text data in an unsupervised fashion and along with the application of self-attention are able to recognize patterns in their training text that is ultimately encoded in the weights of all the layers of the neural network. Honestly, it’s pretty cool.

Contrary to claims made, these models cannot reason, they cannot perform introspection, they cannot plan but none the less they are brilliantly useful for many tasks. They give the illusion that they are performing cognitive functions but it is actually just probabilistic composition of sentences, one token (usually a word) at a time until a complete response is provided. However, the thought that we are close to Artificial General Intelligence (AGI) let alone even sentient AI (AI that can experience human emotions) because we have language models that are able to perform what I would categorize as wide narrow tasks (Wide-Artificial Narrow Intelligence) is to be quite frank, patently absurd.

However that does not mean that generative AI cannot be abused by bad actors, and I assure you that this is an inevitability. Geoffrey Hinton’s departure/retirement from Google and his subsequent conversations with the press are a testament to this. Dr. Hinton recognizes that we may be closer to AGI (not sentient AI) then he originally thought, which is possible but left to be seen as others like Yann LeCun have posited that current deep learning methods are an exit ramp on the journey to AGI, while the 3rd Godfather of AI, Joshua Bengio was a signatory for the Future of Life Institute’s “Pause Giant AI Experiments” open letter. AGI to my best estimates is 40–50 years down the road and that may be optimistic.

However I would like to direct the reader to the topic of synthetic media (which includes images, videos, audio, speech, text and 3D environments) and move off of the discussions around the potential harms of large language models alone (for now.) The topic of synthetic multimedia content has been grossly under-discussed as a collective, this is evident by the recent summoning of the heads of Microsoft, Google, OpenAI and Anthropic by the White House to discuss mitigation of harms and possible legislative oversight for AI. Both tech giants and the generative AI startups they have invested in are principally focused on building large language models and integrating them into their software portfolios, while legislators have kept the conversation on ChatGPT and other chat-bots.

The lack of attention on the topic of synthetic media has made it mostly invisible to the mainstream media and it will remain that way until there are significant harms perpetrated by bad actors, for example the likely interference in the upcoming 2024 elections where nation states and politically aligned bad actors will perform wide-scale public harms with disinformation campaigns. Not giving the possible harms of synthetic media the attention it deserves now is a serious mistake, waiting for it to be featured in the mainstream media after the problem has become pervasive will lead to highly undesirable outcomes that won’t be easily remedied.

That being said, let’s dive right in…

Introduction

Although in the works for nearly a decade, the advancements in generative AI (generative deep learning) over the course of the past 24 months have given rise to a new frontier in the world of digital media: synthetic media. As artificial intelligence (AI) and machine learning algorithms have become more sophisticated, the creation of highly realistic and convincing digital content has never been more accessible. Commercial generative media platforms like MidJourney, RunwayML, Speechify, and deepfake platforms like Reface along with open source(ish) models like StableDiffusion, DreamBooth and others have endowed the public with the ability to easily generate or alter digital media rapidly and with a very low learning curve.

Pope wearing bomber jacket generated using MidJourney v5. (a jacket you can apparently buy now for $179)

This development, however, comes with a myriad of implications for society, including the erosion of confidence in digital media, the potential for misuse by bad actors, the limitations of legislation in addressing these issues, and the increasing importance of authenticity. In this comprehensive analysis, we will delve into the world of synthetic media and explore these critical aspects in depth.

What is Synthetic Media?

Synthetic media refers to the creation or manipulation of digital content, encompassing images, videos, speech, and audio, as well as text and 3D environments generated by AI. One widely used technique to generate synthetic media are Generative Adversarial Networks (GANs), which pit two deep learning models, a generator and a discriminator against each other to generate highly realistic content that is difficult to distinguish from real-life counterparts. Transformers are also a popular choice for serializable data like text and images, and diffusion models have become the de-facto mechanism for the generation of synthetic images which are paired with Neural Radiance Fields (NeRFs), Score Jacobian Chaining (SJC) or Signed Distance Fields (SDFs) to generate 3D assets and scenes.

What are the Net Benefits from Generating Synthetic Media?

The ability to generate synthetic media opens up a range of positive opportunities across various industries and sectors, fostering innovation, creativity, and efficiency. In the entertainment industry, synthetic media can revolutionize filmmaking by creating realistic digital actors, reducing the need for expensive sets, and enabling the seamless integration of visual effects. This technology can also be used to preserve and restore historical footage or even resurrect actors for posthumous performances.

In the realm of education and training, synthetic media can create immersive and realistic simulations, enabling students and professionals to develop skills and knowledge in a safe and controlled environment. For instance, medical students can practice surgical procedures using synthetic 3D environments, while pilots can hone their abilities in simulated flight scenarios with real-time generated elements.

Synthetic media can also contribute to the accessibility of content for people with disabilities. For example, AI-generated speech can be used to convert written text into audio format, making information more accessible for individuals with visual impairments. Similarly, synthetic media can be employed to create sign language avatars, assisting those who are deaf or hard of hearing. Synthetic media can also be used to translate foreign films while preserving the original actor’s voice and update their facial movements to the translated language to provide a realistic regionalized experience.

In the advertising and marketing sector, synthetic media can help create personalized and engaging content tailored to individual preferences, ultimately leading to more effective campaigns. Synthetics (synths) or virtual influencers, powered by generative AI, can interact with consumers in real-time, providing unique and memorable experiences that resonate with the target audience (link to “Rise of the Synths” coming here…)

Furthermore, synthetic media can support scientific research and development by generating realistic simulations and visualizations of complex processes or phenomena that are otherwise difficult to observe. For example, AI-generated visualizations can help researchers study molecular interactions, weather patterns, or even the formation of galaxies.

Lastly, synthetic media can enable creative expression and artistic exploration by providing novel tools and techniques for artists, musicians, and designers. AI-generated music or images can inspire new forms of art, while virtual reality environments can offer immersive experiences that push the boundaries of traditional artistic mediums.

Overall, the ability to generate synthetic media holds immense potential for positive applications across various domains, driving innovation and expanding the horizons of human creativity and knowledge while reducing many of the barriers to entry for media creation through democratization of content creation.

But the same technology that can be used for good can be misused as well as have unintended outcomes.

Eroding Confidence in Digital Media

As synthetic media becomes more realistic and barriers are removed for its quick and low-cost creation, and more and more synthetic media is produced especially that depicting well known people and places but that don’t reflect true events, it will increasingly erode our confidence in digital content. With deepfakes, for instance, the lines between reality and fiction are beginning to blur, which will lead to a greater difficulty in discerning the authenticity of news, social media content, and other digital media forms. This erosion of trust has significant implications for society as a whole:

• Misinformation and Polarization: The spread of convincing synthetic media can exacerbate existing issues related to misinformation and political polarization, making it increasingly challenging for individuals to form evidence-based opinions on various subjects. Boosted by social media and political, philosophical or hate-motivated agendas, the damage will be amplified.
• Credibility Crisis: As deepfakes and other synthetic media continue to proliferate, we may experience a credibility crisis in digital media, where individuals become increasingly skeptical of the authenticity of any content they encounter online.
• Diminished Accountability: The ubiquity of synthetic media may provide bad actors with plausible deniability, allowing them to dismiss genuine evidence of wrongdoing as fabricated or manipulated.

Misuse of Synthetic Media by Bad Actors

The potential for misuse of synthetic media by bad actors is vast with far-reaching impacts, with several notable examples highlighting the risks:

• Disinformation and propaganda: In 2018, a deepfake video of Gabon’s president, Ali Bongo, was used to spread disinformation, ultimately leading to a coup attempt. This example illustrates how synthetic media can be weaponized to manipulate public opinion and destabilize governments.
• Fraud and identity theft: In 2019, criminals used AI-generated voice technology to impersonate a CEO, resulting in the fraudulent transfer of $243,000. This case demonstrates how synthetic media can be used to facilitate financial crimes and identity theft.
• Deepfake revenge porn: Malicious actors may create explicit content featuring someone’s likeness without their consent, causing emotional distress and reputational damage. In a high-profile case, actress Scarlett Johansson’s image was used to create deepfake pornographic videos, emphasizing the harm that synthetic media can inflict on individuals.
• Corporate sabotage: In 2020, a deepfake video of a Russian company’s CEO discussing potential bankruptcy led to a temporary drop in the company’s stock price. This incident highlights how synthetic media can be weaponized to harm businesses and manipulate financial markets.

The Need for Attribution

As we move towards increased fidelity, resolution and realism in synthetic media, the role of attribution will play a crucial part in the production and consumption of media in a number of ways:

• Source Verification: Attribution helps to identify the original source or author of a media content. This will be critical in validating the authenticity of media to determine whether it is real or synthetic and who the original creator is.
• Credit and Recognition: Proper attribution provides recognition to the creators or contributors to a piece of media. This is essential for the respect of intellectual property rights and incentivizing further creation and innovation.
• Accountability: Attribution helps to establish a chain of responsibility. If media content is found to be inaccurate, misleading, or harmful, attribution allows for those responsible to be held accountable.
• Detecting Synthetic Media: Proper attribution can also help in distinguishing between synthetic and authentic media. When a piece of media lacks proper attribution or if the attributed source has a history of producing synthetic or manipulated media, it raises red flags.
• Decentralized Trust: A decentralized system like Secure Digital Asset Containers (SDAC) will address this need by validating identities that are used to validate digital media and reference validators who put their own reputations on the line to assure others that media assets published in the network are in fact legitimate and can be attributed to a trusted source.
• Content Authenticity: Content Authenticity Initiative (CAI) has been proposed by Adobe, Twitter and The New York Times. This initiative aims to develop an industry standard for digital content attribution, allowing creators to securely attach attribution data to content they produce. This digital provenance, embedded in the file, would then travel with the content and provide a means for end users to verify the source of the media.

The need for attribution in the production of media cannot be overstated. As synthetic media technologies become more advanced, so too must the methods and tools we use for attribution and verification.

Validating Authenticity

In light of the growing challenges posed by synthetic media, authenticity will become increasingly important for digital content. As trust in digital media erodes, consumers will seek out reliable and verifiable sources of information, and organizations will need to prioritize transparency and fact-checking.

There are a few methods and strategies we can employ for handling synthetic media, along with several current and emerging approaches that can be employed to promote authenticity and combat the spread of synthetic media or mitigate its effects:

• Digital watermarking: Embedding a digital watermark into original content can help verify its authenticity and source, making it more difficult for bad actors to manipulate or misuse the content.
• Distributed ledger technology: Leveraging the immutability of blockchain or directed acyclic graph networks, it is possible to create tamper-proof records of content creation and ownership. This can help verify the authenticity of digital media and restore confidence in its credibility.
• AI-based detection tools: Developing AI-driven tools that can detect synthetic media with high accuracy will be crucial in flagging and removing manipulated content. The caveat here is that no tool is 100% accurate and as AI-generated digital media techniques improve, the ability to detect content will become less and less viable.
• Media literacy education: Teaching media literacy skills, including how to recognize synthetic media and evaluate the credibility of digital content, can empower individuals to make informed decisions about the information they consume. This includes teaching critical thinking in the classroom, an area that is wholly under-served in the education system. The failures here are evident when we look at the results of the effects of today’s social media landscape.
• Industry collaboration: Encouraging collaboration between tech companies, content creators, and policymakers can help develop a comprehensive approach to address the challenges posed by synthetic media and prioritize authenticity in the digital landscape. Ideally this collaboration would lead to the development of industry standards that can be employed to ensure that authenticity and integrity are maintained throughout the lifecycle of digital media from creation to presentation.

Navigating Copyright Issues

Navigating copyright issues with synthetic media is a complex and evolving area of the law. The key to this lies in understanding the foundational principles of copyright law, and how they apply to synthetic media, which can include deepfakes, artificial intelligence-generated art, music, writing, and other forms of content.

Since not all use of synthetic media is inherently bad, it makes sense to take an informed approach to generating synthetic media, especially if one intends to use the outputs for commercial purposes. However being a user of a generative AI model and not knowing what data it was trained on, can lead to unanticipated legal challenges related to the output of generative models. For instance if you generate a piece of AI art, but it looks nearly identical to an existing piece of art, or you generate a screenplay that appears to borrow substantially from prior copyrighted work, you may run into legal issues especially if you assert any claims over the generated work or use it for commercial purposes.

Here are a few ways one might navigate copyright issues with synthetic media:

• Originality: Copyright law typically only protects original works of authorship. If a piece of synthetic media too closely resembled an original copyrighted work from which it was derived (through training and generation), then it may infringe upon the copyright of the original work. If your synthetic media is truly original, it will generally be safe from copyright infringement claims.
• Fair Use: This is a defense to copyright infringement in certain jurisdictions like the U.S. Fair use allows limited use of copyrighted material without permission from the original author for purposes such as criticism, parody, news reporting, research and education, or commentary. The boundaries of fair use are a bit blurry and can depend on the specific circumstances of the use, but it’s one potential avenue to navigate copyright issues.
• Licenses: One of the safest ways to use copyrighted material is to obtain a license from the copyright owner. This usually involves paying a fee and agreeing to certain terms of use. Licenses are particularly important when dealing with synthetic media that heavily relies on copyrighted works.
• Public Domain: Works in the public domain are not protected by copyright, and can be used freely. If a piece of synthetic media is based on works in the public domain, it will generally not face copyright issues.
• De Minimis Use: This legal doctrine posits that trivial uses of copyrighted material are not infringement. However, what counts as “trivial” can be subjective and may depend on the specific circumstances. I would not recommend you rely on this as a defense against any claims that may be made against you in using synthetic media as it is an unproven position.
• Lawful Exceptions: Some jurisdictions have lawful exceptions for the use of copyrighted material without permission, such as for the purpose of caricature, parody, or pastiche.
• Collaboration: Collaborating with the original author or copyright holder can also be an option. This can lead to shared ownership of the copyright and can mitigate copyright disputes.

However, it’s important to note that the legal landscape around synthetic media and copyright is still evolving. Recently the US Copyright Office declined to approve an application to copyright AI generated images as well as a graphic novel generated by AI. Laws and precedence will be set as the technology advances and claims work their way through the court system. It’s also important to note that different jurisdictions may have different rules. Therefore, it’s a good idea to consult with a legal professional when navigating these issues.

The Limitations of Legislation

While governments and organizations are beginning to recognize the dangers of synthetic media, legislation alone is unlikely to be sufficient in mitigating its harms. The following factors contribute to the limitations of legislation in addressing synthetic media:

• Global nature of the internet: The internet transcends borders, making it difficult to enforce laws across different jurisdictions. As a result, individuals and entities may exploit legal loopholes by operating from countries with lax regulations.
• Speed of technological advancement: The pace at which technology advances often outstrips the ability of lawmakers to create and implement effective regulations. By the time legislation is put in place, the technology may have already evolved, rendering the laws obsolete.
• Freedom of speech concerns: Legislating against synthetic media may raise concerns about freedom of speech and expression, particularly when it comes to satire, parody, or other forms of creative expression that utilize synthetic media in a non-malicious manner.
• Detection and enforcement challenges: Identifying synthetic media and attributing it to specific creators can be difficult, making enforcement of legislation problematic.

Effective Detection

Detecting deep fakes and synthetic media up until recently has been viable without tools, just using a discerning eye, however we are approaching a point, a Matrix moment if you will, where digital media no longer can be relied upon at face value to reflect reality.

As synthetic media technology improves and becomes more capable of producing media that is practically indistinguishable from reality, the task of detecting synthetic media becomes increasingly difficult. Here are several limitations of detecting synthetic media:

• Quality of Synthetic Media: The sophistication of AI technologies is approaching a point where it can nearly generate ultra-realistic images, audio, text and videos and being able to create synthetic media that is indistinguishable from real media is in the not too distant future. This will make it harder for both humans and automated detection systems to distinguish between synthetic and real media.
• Constant Evolution: AI models used to generate synthetic media are continually evolving, and with each iteration, they correct for flaws that could have been used as a telltale sign of synthetic media in the past. As such, detection mechanisms have to continuously adapt to keep up with the rapidly advancing technologies used to create synthetic media.
• Volume of Data: With the explosion of digital content creation, the sheer volume of media that needs to be scanned for authenticity poses a significant challenge.
• Lack of Legal and Ethical Frameworks: As of now, there’s a lack of robust legal and ethical frameworks to regulate the use of synthetic media. While tech companies and researchers are working on tools to detect synthetic media, their efforts are often hampered by the absence of universal standards and guidelines. (see article on Responsible AI.)
• Proof of Authenticity: It is challenging to establish an immutable chain of custody for digital content, complicating the task of proving a piece of media’s authenticity.
• Invasion of Privacy: Some methods of synthetic media detection might require access to private data (e.g., original pictures or videos, geolocation information), which can lead to potential privacy concerns.
• Cost and Resources: Developing, maintaining, and improving synthetic media detection tools require significant investment in terms of time, money, and computational resources.

Despite these challenges, there’s a significant amount of research and development focused on improving the detection of synthetic media. Techniques such as digital watermarking, distributed ledger-based verification, and advanced machine learning algorithms are being explored and developed to counter the threat of indistinguishable synthetic media.

Conclusion

Synthetic media is a powerful and emerging technology that carries both benefits and risks. As it continues to develop, it is crucial to foster an environment that encourages responsible use and prioritizes authenticity.

By doing so, we can harness the potential of synthetic media while mitigating the risks it poses to our digital landscape. Only through a multi-faceted approach that combines technological innovation, legislative action, and public education can we navigate the complex challenges posed by synthetic media and maintain trust in the digital realm.