Top Predictions and Trends for Generative AI in 2024

18 min readDec 28, 2023

Generative AI still has a long way to go!

ChatGPT Prompt: “generate an image using Dall-E 3 of Nostradamus predicting the future of generative AI.”

At the end of 2022 I put forth my predictions for generative AI technology, policy, commercialization and some predictions on innovations and challenges that would arise in 2023. After reviewing those predictions, all have come to fruition in 2023 including the development of a 10-20B parameter image generation model, the trend of conventional search being displaced (not replaced) by chat, as well as the introduction of legislation to curtail generative AI misuse, the concerning issue of synthetic child pornography and the rise of synthetic influencers. My 2023 predictions can be found here.

For 2024 I will assume the same general format with new predictions and the identification of trends that will help shape generative AI over the next year. 2023 was the year the mainstream was introduced to generative AI and users experimented directly with generative AI applications like ChatGPT, however 2024 will be the year of enterprise and industry adoption of generative AI. Where businesses will see meaningful impact to their bottom and top line revenues and more organizations will shift to an AI-first mindset.

This trend will be pushed forward by a more realistic and informed approach towards use case identification, stabilization and safety of generative models and a shift towards smaller more specialized open-source models operating in ensembles that can be hosted by organizations in their own cloud environments. The biggest innovations will be in generative AI applications, whether application and agent frameworks or creative uses of generative AI higher up in the stack to solve business problems.

Generative AI has a long way to to go yet and there are still many exciting innovations on the horizon. So let’s dig into what I see happening with generative AI in business, the enterprise, technology development, applications, policy and responsible AI development in 2024.

In Business

This is where generative AI will be most impactful in 2024 as organizations have begun to understand how generative AI works and how it can make an impact in their organizations, whether improving existing products and services or creating new revenue opportunities. The flurry of investments in generative AI startups we saw in 2023 will tone down (already down 23% in Q3/23). Especially after investors saw their investments in startups that built moatless applications using thin-wrappers over OpenAI’s GPT-x models (Jasper, Copy.AI) or on top of widely available open source alternatives (Character.AI) become indefensible. This was an obvious outcome that I posted about in September of 2022 and referenced in my 2023 predictions.

Enterprise and Industry

This may seem obvious, but a growing number of organizations will actively explore and integrate generative AI into their applications and services, significantly impacting various business functions (see use cases in the “Inside The Enterprise” section. Some to great success and many others to complete and utter failure. I would put the ratio of success to failure at 3:7. With only 3 out of every 10 generative AI projects making it to production with longevity in 2024, most will be experiments that don’t see production traffic.
There still exists a vacuous space of experienced generative AI consultants able to effectively accompany enterprises on their generative AI journey. Although the consulting space for generative AI will continue to grow, there will remain a small but growing number of highly qualified consultants able to move customers into highly differentiated spaces in generative AI.
User interfaces for generative AI applications will transition from being “hobbyist” (lots of configurable parameters, think Automatic1111 for Stable Diffusion) and “basic” (no configurable parameters) to user-friendly and consistent with familiar user experiences. Most will be hidden behind existing applications or sidecar with a “chat” experience as generative AI becomes less visible in applications.
The production of high-performing open-source AI models will challenge proprietary solutions, democratizing access to generative AI technologies. Many will continue to make claims that they outperform GPT-4 on particular tasks but no single model will match across all industry benchmarks (which remains a sock drawer of non-standard and often non-scientific evaluations against non-standard metrics).
The increased viability of open-source models will be complimented by a rich eco-system of open-source application frameworks, databases, libraries and applications that will amp up interest in open-source solutions and initiatives like the Linux Foundation’s Generative AI Commons. Driving more companies to move their projects into open source.
The first “model” to match GPT-4 across the board for performance of all tasks will in fact be multiple models (which GPT-4 clearly is) running in an ensemble of experts architecture. Anyone still holding onto the dream that OpenAI’s GPT-4 is a single highly parameterized model behind an API will be sorely disappointed in 2024. This deployment architecture is referred to as a Large Model Systems (LMS) as it is comprised of not only multiple deep generative models and conventional machine learning models but also supporting infrastructure like API gateways, databases, authentication systems, and so forth.
The term “Frontier Model” will fall out of fashion, as the hilltop that big tech companies have been sitting on will slowly be within reach of smaller competitors.
The most innovation will take place in AI applications, building on top of generative models with growth in application frameworks and features like prompt stores, prompt cacheing, prompt translation and other platforms that simplify, bring consistency, interoperability and portability to generative AI application performance.
Corporations will leverage generative AI to innovate in product and service offerings, enhancing customer experiences and opening new revenue channels. Although it will generally be a slow roll to production.
Large Language Models (LLMs) will give way to Medium Language Models (MLMs) and Small Language Models (SLMs) as a trend towards developing smaller, more efficient generative AI models will emerge, focusing on optimizing performance with reduced resource requirements.
While Retrieval Augmented Generation (RAG) will remain popular, its unique advantage will diminish as it becomes a standard feature in AI systems.
Fine-tuning will become more accessible through model management frameworks and will fuel a market of domain-specific smaller language models, some for public consumption and others kept within the IP portfolio of companies due to privacy concerns and competitive edge.
The availability and sophistication of non-English language LLMs will increase, broadening the global reach of generative AI technologies.
Federated learning will see notable advancements, especially in highly-regulated sectors, enabling more secure and compliant AI deployments. This will be especially valuable for organizations who will not or cannot share data but collectively want to train highly capable models for their industry (training data memorization will still be an area of concern.)
The exploration of LLM-agent frameworks will intensify, with organizations experimenting to leverage their potential in various applications. This will spur more development around open source projects like AutoGen that are more specialized agent frameworks, unlike LlamaIndex and LangChain which are generalized LLM application frameworks with unnecessary features.
The proliferation of user-friendly AI development platforms will empower a wider range of users to fine-tune and customize AI model systems. With a shift towards nocode and lowcode platforms like LangFlow and tighter IDE and notebook integration like Jupyter-AI.

Mainstream and End Users

The initial hype around generative AI will transition to more normalized expectations as the technology becomes more integrated into everyday applications.
The media, companies, doomsayers and anyone looking for 15 seconds in the limelight will continue to push the “AGI” button, but this should be a clear indicator that there is a motive behind the fearmongering and sensationalism of AGI. We are still a long ways away from seeing AGI made a reality and it most certainly won’t come from the current state-of-the-art in deep learning (ie. LLMs).
Generative AI will become increasingly embedded in user interfaces, enhancing user experience by making advanced AI functionalities more accessible and user-friendly and less “in your face”. This will be most evident with the integration of LLMs with Microsoft Office and Google Workspaces, and the Windows operating system.
Public concerns regarding AI’s potential harms will evolve into a more balanced understanding, acknowledging both the benefits and challenges of AI technologies. However Responsible AI will still be a guiding light for those working in generative AI development and the true harms of the misuse of generative AI cannot be ignored.
Hyper-personalization through generative AI will attract significant attention and investment, leading to innovative applications in marketing, entertainment, and personal services such as dynamic personalized ad generation and adaptive media.
Marketers will explore means to get brands and other marketing materials into training datasets in order to get their products, services and brand names in front of users of chat-based LLM-backed services in a crafty way much like SEO with conventional search. API services may explore injecting promoted content into model outputs.

Startups, Venture Capital and Investments

The emphasis on open-source and open-science approaches in AI will intensify, attracting investor interest in startups prioritizing these modes of operation.
Venture capitalists will re-calibrate their investments as basic “thin wrapper” generative AI applications become commoditized. The same features offered by companies like Jasper will be fully integrated into Microsoft Office and Google Workspaces, leading to pivots and liquidations.
Many early generative AI startups without “moats” will fail wiping out 10s of millions in early VC investments.
The investment climate will become more discerning, with a greater emphasis on startups that offer unique, sustainable competitive advantages.
Startups focusing strictly on hosting open-source AI models will face increased competition from larger cloud service providers, necessitating innovation to remain relevant.
Several startups will venture into the trusted AI space looking to solve the problem of trustworthiness of generative models including model signing and lineage tracking.
VCs will actively seek out startups that can offer enterprise-ready solutions without all of the bloat and instability found with many of the current open-source solutions.
The ongoing GPU shortage will spur investment in alternative computing architectures for AI like RISC and alternatives to GPUs like ASICs and FPGAs.
Investors will look startups that are innovating in the space of Off-The-Shelf (OTS) generative AI solutions that customers can operate in-house. Platforms that reduce the barrier to entry and provide a wide assortment of generative AI features without the need to manage and maintain the underlying technologies.

Big Tech

Nvidia will continue to push towards becoming a hyperscaler, and although they possess the resources to build out the infrastructure, they will face stiff competition from the incumbents and are not likely to make much headway in 2024.
Although not having a history of contributions to generative AI research like their competitors, AWS will come out on top as the hyperscaler seeing the most growth for generative AI workloads due to its existing user base and their approach to offering tiered solutions to access both generative AI API services and open-source models (Bedrock, SageMaker and IaaS/PaaS.)
A collective effort by competitors in silicon will attempt to challenge Nvidia’s market dominance, pushing for more research and innovation on top of alternative architectures from AMD and Intel.
Google, Microsoft, Nvidia and Amazon having made investments and established partnerships with generative AI-focused startups in 2023 will begin to bring talent in-house to de-risk dependency on external relationships.

Inside the Enterprise

2024 will see organizations become more comfortable with generative AI, invest more money in training and hiring staff capable of working with generative AI technologies and with the reduced frenzy around generative AI, organizations will take a more methodical and level-headed approach to applying generative AI in their organizations.

In addition to the “Industry and Enterprise” overall trends there are some specific areas inside the Enterprise that will take place over the next 12 months that we should not pass over related to adoption and operations.

Generative AI Adoption

Organizations will focus on many of the same key use cases from 2023, including chatbots, application and customer service assistants, knowledge access (RAG), anomaly detection use cases, software development assistance, marketing and sales collateral and blogs, and the multitude of language assistant tasks provided by LLMs (translation, formatting, ideation, templates, etc…)
Companies will continue to be concerned about the risks of using generative AI platforms and models, and will seek greater transparency. Especially those models that do not provide model lineage and supply data sets for full transparency. Governments will equally push for greater transparency.
Organizations will continue to prefer black-box solutions like GPT-4 and Claude 2 but will more actively evaluate open-source options as their performance increases and they become more comfortable with the technology.
We will see a heavier focus on adopting guardrails for generative AI models, using classifiers to identify potentially bias and harmful content or prompts that are exploitative in nature on both ingress (prompts) and egress (responses) for generative models.
Generative AI governance will become better defined and standard governance models will be proposed that will address data and model lineage, privacy and security as well as conformance to regulations.
More organizations will look to hire Chief AI Officers in order to give them strategic advantage over their competition and help facilitate innovation and integration of generative AI into their business. Similarly more companies will stand up AI Centers of Excellence to bring cross-organization stakeholders and executive leadership together to become AI-first organizations.

Infrastructure and Operations

There will be a notable increase in specialized, domain-specific open-source models reaching production-grade status. The trend of trying to outmatch on generalized models will dissipate as the performance curve starts to level off. However a major architectural innovation could change this substantially.
The ‘Ensemble of Experts’ approach will gain traction as the preferred architecture for deploying and managing diverse generative AI models inside of an organization’s own cloud infrastructure, however the uptick will not be drastic in 2024 as concerns still loom about unintended consequences with self-managed open-source models, and the workforce remains rather inexperienced when it comes to generative AI technology management.
Multi-modal guardrails will become standard in generative AI deployments, ensuring better control and alignment with ethical standards. Organizations will prepare their organizations to work with all types of multimedia safely and responsibly.
GenOps will emerge as the dominant framework for managing generative AI operations, replacing the more narrow LLMOps approach, as organizations ensure that their investments are future-proofed and protected against redesign and re-engineering.

Generative AI Technology

The number of research teams in academia, and the private and public sector has increased substantially. This will lead to more innovations in 2024 but they will for the most part be incremental enhancements. It is possible that a major architectural innovation will be developed but with the bulk of the focus on enhancing transformers, the odds are fairly low that we will see something on par with “Attention is all you need.”

Organizations will continue to offer less and less information about their research when they release papers, and some may refrain from releasing their research at all, breaking with the principles of open science that has been at the core of AI research for decades.

Model Performance

AI researchers with renewed investment by big tech companies partnering with their academic institutions will continue to innovate, developing new methodologies and techniques to optimize the performance of generative AI models, focusing on efficiency, consistency and scalability.
Techniques like LongLoRA and QLoRA will lead the way in parameter-efficient fine-tuning (PEFT), allowing for more adaptable and resource-efficient model training. Perhaps a derivative method will be developed that provides marginal optimization.
Sparse attention and flash attention mechanisms will inspire newer methods of optimizing training, improving the computational efficiency of generative AI models based on the transformer architecture, with a particular focus on optimizing GPU memory storage and bandwidth, as well as computational resource utilization.
The use of Grouped Query Attention (GQA) and Sliding Window Attention (SWA) will see expanded use with open source transformer-based models, enhancing the inference performance.
The application of ‘mixture of experts’ with large language models will continue due to its inference performance benefits, but with no appreciable benefits for GPU memory utilization, widespread adoption will only be realized with innovations in memory efficient optimizations.
RLHF will remain the predominant method for model alignment, with Direct Policy Optimization (DPO) increasingly replacing Proximal Policy Optimization (PPO).

Generative AI Research

A shift towards CPU inference and edge-based AI processing will be observed, driven by the need for more decentralized and accessible AI applications on end-user devices like laptops and mobile phones.
Innovative model architectures not based on transformers will emerge, offering new possibilities in generative model design and performance.
Interest in multi-modal AI models will grow, with a focus on bi-modal integrations of text and images or text and audio, but will be tempered by deployment architecture techniques like ensemble of experts which can functionally perform many of the same tasks without the burden of higher parameter counts and GPU memory consumption.
Researchers will look for more human-like methods of intelligence that aren’t currently satisfied with next-token-predictors, specifically reasoning, hierarchical planning, long-term memory and introspection.
Online and active learning (with use of model-based oracles) will influence the development of learning methods for generative AI models to keep them up-to-date, aligned and prevent catastrophic forgetting without having to perform batch fine-tuning.
On-device learning will become a focused area of research for small models capable of running on end-user devices. RLHF will be applied to a constrained environment in order to enhance the performance of device-based models informed by individual preferences.
Methods to forget specific details or knowledge captured in models without causing catastrophic forgetting will be explored, however due to the sensitivity of parameter adjustments this will be a difficult task.

Applications

Generative AI applications will be the biggest space for innovation in 2024, as interest shifts towards better understanding how to get the most value out of generative models. Prompt engineering will become another skill in the repertoire of application developers, and not a job unto itself.

We will see improvements in all the modalities, with the biggest advancements in audio and music generation as well as voice synthesis. Video and 3D generation will continue to move along but at a much slower pace.

Image Generation

Image generation models will continue to improve in quality, with some achieving levels of photorealism that are indistinguishable from real images.
Inference times will decrease to the point where frame-by-frame style transfer can be reasonably achieved with commodity hardware in real-time.
A greater emphasis will be placed on interactive and iterative editing capabilities with image generation models, enhancing user control and creativity.
Challenges related to text generation in images will be largely solved, leading to more accurate text in image outputs.

Video Generation

Progress in video generation will be incremental, with slight improvements in generation times but no breakthrough advancements.
The issue of temporal consistency needs a new innovation, which may come to fruition, but for now 5–10 second videos will remain standard and will still have consistency issues.

Audio Generation

The field of audio generation, including synthetic voices, sound effects, and music generation, will see significant advancements in quality and versatility.
Voice replication technology will advance to the point where it becomes virtually indistinguishable from actual ground truth human voices.

3D Generation

Incremental progress in 3D generation will be made, with slight improvements in quality and generation time, though realism will continue to enhance.
Hybrid deep learning methods for 3D asset generation will still prevail, using radiance fields and gaussians.
Its possible that another enhancement to NeRF could reduce long inference times to generate 3D textured models, but seems unlikely.
More innovations will be centered around Gaussian Splatting rather than NeRF, leading to incremental enhancements.

Software Development

The integration of code generation tools with IDEs and notebooks will deepen, leading to greater reliance on AI-assisted coding and real-time code insertion.
Companies will increasingly utilize generative AI for automating and enhancing software testing processes, including unit and integration testing.
IDE co-pilots will continue to provide value and become more capable at producing the desired outputs based on developer prompts. Developers are still a long way from being replaced and will work with generative AI technology to achieve their programmatic outcomes.

Across Industries

There will be major innovations and movement in industry specific applications of generative AI as well as AI literacy and training of the workforce on generative AI technologies. Creative uses of generative AI will be applied effectively in healthcare and medical research and in social media to create a new set of “synthetic influencers”.

Education and Training

The market for AI-related courses will remain saturated, with limited substantial new innovations to drive out new differentiated content.
Challenges in AI literacy will persist, with slow adoption of AI education in public education curricula.
An increase in Open Source Program Offices (OSPOs) at universities will encourage more open-source research and collaboration resulting in more open science and open-source projects.
Debate and confusion over what constitutes ‘Open-Source AI’ will continue, with some projects being criticized for open-washing.
Generative AI will be increasingly used to create personalized educational experiences, adapting to individual learning styles and needs. Startups will seize upon generative AI capabilities to develop more personalized Learning Management Systems (LMSs).
The development of AI-powered tutors will progress, offering real-time, interactive assistance in virtual learning environments.

Social Media

The trend of creating synthetic influencers using generative AI will grow as a new avenue for monetizing large online followings.
Social media platforms will become increasingly saturated with AI-generated content, leading to audience fatigue with repetitive and generic outputs.

Industry-Specific Applications

Generative AI will continue to make significant inroads in medical and drug research, offering innovative solutions and accelerating discovery processes, mostly led by work at Google DeepMind.
The development of open-source models focused on scientific and mathematical applications will increase, fostering broader innovation and collaboration.
AI applications in logistics and supply chain management will become more advanced, using both generative and predictive models to optimize processes and reduce inefficiencies.

Responsible AI and Trustworthy AI

The concerns around bias, transparency, copyrights and unintended consequences are not going away any time soon. The courts will preside over numerous cases of purported copyright violations, demonstrable harms due to generative model outputs, and cases of impersonation where unauthorized likeness of celebrities will be used without their consent.

Some governments will rush in legislation, while others will take a sit back and see approach. Concerns about environmental impacts of training and running generative models will grow and the FUD (Fear, Uncertainty and Doubt) around generative AI will concern workers that their jobs are in jeopardy.

Security, Privacy, and Safety

Trends in malicious use of AI for creating harmful or deceptive content will persist, necessitating advanced detection and prevention methods.
Generative AI will be leveraged to interfere in the 2024 elections by politically-affiliated organizations in the U.S. and by nation states. The misinformation campaigns will be mounted at scale with fake accounts, synthetic images and content generated by LLMs in order to confuse and anger voters to push them towards a particular candidate.
Efforts to establish content attribution and protect the rights of original content creators will intensify in response to the proliferation of AI-generated content. The New York Times vs. OpenAI case will set the precedence for how generative AI content that is memorized from training data will be viewed in the eyes of U.S. law.
The demand for tools to differentiate between human-created and AI-generated content will grow, driven by concerns over authenticity and trustworthiness. This will spur a bunch of new startups and industry initiatives.
The potential for generative AI to be weaponized by nation-states will become a pressing concern, with implications for global security and political stability.

Ethical AI Governance and Policy Development

Legal and regulatory responses to the training of generative models on copyrighted materials will evolve, with potential decision reversals or new laws classifying outputs of generative models as derivative works.
The U.S. will delay the enactment of restrictive AI legislation to maintain its competitive edge in AI innovation, contrasting with approaches taken in Europe.
An indivdual will sue OpenAI for the right to be forgotten from their foundational model service. This will set off a hot debate about how the right to be forgotten can be applied to LLMs where its not easily achieved.
The music industry will experience the challenges of authenticity posed by generative AI, with artists and industry bodies taking a more aggressive stance against potential copyright infringements and impersonation of creative works.
The development of an industry standard ethical AI framework will gain momentum, as the need increases to address growing concerns around bias, fairness, transparency, and accountability with generative AI models.
International standards and regulations for generative AI will be pursued, aiming to establish consistent and ethical practices across different countries and industries pushing for significant international collaboration.

Human-AI Collaboration and Augmentation

There will be a growing trend of using generative AI to augment human creativity and productivity, rather than replacing human roles, especially in creative and design industries.
The adoption of generative AI in creative fields will increase, with AI being used as a collaborative tool for conceptual design and creative exploration.

Environmental Impact of AI

The environmental impact of training and operating large-scale AI models will become a more prominent concern, leading to innovations in more energy-efficient AI algorithms and infrastructure.

Cybersecurity and AI

AI-driven cybersecurity solutions will emerge as critical tools for predicting and mitigating potential threats, using anomaly detection with generative models and enhanced pattern recognition.
The rise in generative AI-powered cyber attacks fueled by LLM-based agents will begin to surface. Initial methods will be crude but will lead to more sophisticated methods able to penetrate attack vectors and automate social engineering attacks.

As we look towards 2024, the landscape of generative AI is poised for transformative changes across a multitude of sectors. While the initial hype surrounding this technology is expected to stabilize, its integration into mainstream applications will deepen, making these advanced tools more accessible and ingrained in our daily lives. This transition will not only redefine how businesses operate but also how we interact with technology on a personal level.

In the enterprise and industry sectors, the adoption of generative AI is set to accelerate, driven by the development of more efficient, powerful, and accessible models. This growth will be complemented by significant advancements in infrastructure and operations, enabling more sophisticated and ethical deployment of AI technologies.

The investment landscape will also undergo a shift, focusing more on sustainable and innovative AI solutions that offer a distinct competitive edge that will make them difficult to unseat. As the AI market matures, discernment and strategic vision will be key for investors and companies alike.

In the spaces of education, social media, software development, and industry-specific applications, generative AI will continue to push boundaries, creating new opportunities and challenges. These advancements will be accompanied by an increased focus on security, privacy, and ethical considerations, underscoring the need for responsible and thoughtful integration of AI technologies.

As we embrace these changes, it’s crucial to maintain a balanced perspective, recognizing both the immense potential and the challenges that come with such rapid technological evolution. The future of generative AI is not just about process efficiency and creative capabilities; it’s also about shaping a world where this technology enhances human capabilities, fosters innovation, and contributes positively to society.

The journey ahead is as exciting as it is complex, and staying informed and adaptable will be key to keeping up with the pace of innovation in generative AI in 2024 and beyond.