Generative AI is the hottest area among VC circles with hundreds of startups being launched at breakneck speed. It started with ChatGPT, the generative AI app from OpenAI, capturing the attention of the everyday to knowledge workers to technology experts, reaching 100M users by January 2023 — just two months after its Nov 2022 release. OpenAI then released GPT-4, its new and improved large multimodal model in March 2023. GPT-5 is expected by the end of the year. In reaction to competition from OpenAI’s ChatGPT, Google released Bard in March 2023. Microsoft announced AI Copilot for Microsoft 365 and in April 2023, Meta released its Segment Anything Model (SAM) that can “cut out” any object in any image with one click. AI has been around for quite some time, but the release of ChatGPT and Generative AI apps has made it more visible than ever. GPT -4’s ability to accept images as inputs is a game changer as this capability broadens use cases and the types of applications that can be built on top of the model. Past models such as GPT 3.5 have been text-only, but with the release of GPT-4, a multi-modal model, we are increasingly seeing a shift towards models that can accept multiple types of inputs (image, video, text, audio). The ability for models to interpret and generate responses based on multiple modalities leads to more accurate and context-relevant outputs and ultimately unlocks a world of possibilities for startups building on the app layer. We at Service Ventures believe that Generative AI applications such as ChatGPT will have a significant impact on the way people do their everyday work. It has the potential to increase our productivity exponentially over time.
Generative AI could be easier for various industries to leverage for business use cases given the heavy lifting of training an AI model is done by OpenAI and others providing the foundational models. Given AI models are extremely complex and resource-intensive, costing businesses millions of dollars and years to train, and given the easy accessibility for third party developers to build on top of a trained AI model, it is difficult for new AI foundational model providers to spring up and reach parity unless they have the capital and resources to do so. So, we think OpenAI might remain the dominant player given its lead in Natural Language Processing (NLP) and smaller players could appear with a focus on highly specialized models for specific use cases. This phenomenon could continue to play out across modalities from text to image and video and beyond. Like how companies today use a combination of AWS, Azure, GCP, IBM Cloud in their tech stacks as a form of risk mitigation and diversification, we could see startups moving towards using a combination of AI models, from horizontal, broad-based models to those that are more vertical-specific. But the evolving ecosystem needs to be evaluated critically to understand the opportunities for startups.
The Foundational AI Technology
Let’s start with the foundational AI model. ChatGPT relies on a Large Language Model (LLM) – a type of AI model to produce well-strung words together, coherent responses to various questions. A Large Language Model (LLM) is a deep learning algorithm that can recognize, summarize, translate, and generate content based on the knowledge gained from other data sets. AI research has been around for a while, but recent Deep Learning algorithms have been made possible by the vast amount of digital data available on the web, by advances in computer power, and by the development of GPUs (Graphic Processing Units). Some foundational AI models are open to use, meaning the source code is available to all or some members of the public while others are closed proprietary source codes, inaccessible to the public. AI models can be multi-modal, meaning they work with more than one content/data format (text, image, video, audio), while others can be mono modal. LLMs are typically based on transformer architecture – a specific approach to build an AI model and trained using reinforcement learning. All LLMs (open or closed) are capital-intensive endeavors, taking up a lot of time and money to build. A Big Tech company or a very well-funded startup can build these models.
Open vs Closed AI Models
Open LLMs offer many benefits for startups as they allow teams to leverage an already existing models and code, accelerating the pace of experimentation without the need for deep expertise in LLMs and Machine Learning. This is flexible and cost-efficient for startups to test, experiment, and launch AI products quickly. But subpar AI outcomes resulting from incorrect facts and stale training data may become problematic, especially in high-accuracy verticals such as healthcare and banking. Closed, proprietary LLMs, on the other hand, could give better privacy and increased accuracy due to fine-tuning on new/specific data sets and tasks, and better performance as models can be optimized for specific hardware of the vendor. However, with closed LLMs, startups are tied to the LLM developer and from a cost structure perspective, API costs can negatively impact business model sustainability. Startups founders usually do not get to know and understand the true magic behind those pre-trained closed AI models. There is also the risk that the LLM a startup is using ceases to exist or is no longer available due to regulatory crackdowns or other factors outside the startup’s control. Switching costs could be significant between open and closed models. When switching between models, integration, and compatibility are key factors to consider — it may be time consuming and expensive for a startup to adjust its existing infrastructure, software, and apps to work with a new underlying AI model. A balance needs to be maintained between the speed and innovation of open models with the privacy and accuracy of closed proprietary models.
The AI Tech Stack
There are multiple layers in the AI stack for startups to launch and thrive in this space, although the model layer of the AI stack has garnered a lot of attention in the past year from OpenAI.
1) Infrastructure Layer: It is the cloud and HW infrastructure that facilitates the training of LLM in the first place. Providers in the infrastructure layer could again be the true winners at this stage in the Generative AI revolution, given the amount of computing power needed to train LLMs. Cloud providers such as GCP, AWS, Azure, IBM and semiconductor manufacturer AMD, Nvidia, Cerebras are cashing in at this layer.
2) AI Model Layer: This is the foundation AI model layer, which includes open-source and closed-source large language models (LLMs) from OpenAI’s GTP 4 to Anthropic to Cohere to Stability and Google’s PaLM.
3) Model Ops Layer: The model ops layer allows developers to optimize, train and run their models more efficiently. DeepInfra, MLflow, Hugging Face, OctoML and Modal are examples of startups in this layer.
4) Data Management Layer: The data management layer consists of Ai specific databases for information storage such as Vector Databases and Data Manipulation infrastructure. Vector databases can store and index large amounts of data, providing a better way to store and organize data than traditional databases. Several players in this layer include Pinecone, Chroma, Activeloop etc. Data manipulation layer makes data AI-friendly, with building blocks that connect the messiest data to the AI models. Startups in this category include Airbyte, MotherDuck, and Unstructured.
5) Tooling Layer: The tooling layer enables developers to build foundational model applications more quickly. Examples of startups operating in this layer are PromptLayer, Fixie, Gradio, LangChain and HumanLoop. This layer is especially sticky as it is embedded in the designers’ workflow. Switching costs are high, and this layer is critical to making the foundation model more accessible and user-friendly to a wider range of developers.
6) Application Layer: This is the layer that most everyday consumers will interact with. There has been an explosion of companies in this area such as Jasper, Runway, Harvey, and Tome. The application layer development is typically quick and less costly; however, product differentiation could be hard, and retention could be difficult.
Generative AI Native Apps vs Generative AI Enhanced Applications
Generative AI Native Apps are built directly on top of an LLM and generate content without needing explicit instructions other than simple “prompts” — examples include Jasper, Harvey, RunwayML, Fireflies.ai. Generative Native Apps can benefit from low cost to build as they rely on APIs to external LLMs and they don’t at the onset need to invest large amounts of capital to build their own models. This allows a faster time to market, like what we saw with cloud-native companies that didn’t need to spend the time and capital to build on-premises infrastructure. Like cloud computing, Generative Native companies can dial up and down their use of external LLMs as they adapt to workload and demand variability. These apps could manifest as horizontal applications.
Generative AI Enhanced products on the other hand are already existing SaaS products such as Notion and Canva that are beginning to integrate AI or LLM features into existing products. At Service Ventures, we think if a company does not have LLM integrated in their existing product within the next two to three years, they could be at risk. Integrating Generative AI into existing workflows can be challenging, increase development costs and time, and potentially result in compatibility issues that may or may not affect the user experience.
AI Native Apps can be segmented by vertical applications (Banking, Accounting, Sales & Marketing, Legal, Productivity), by modality (text, image, video, audio), or by form factors such as desktop, mobile, plug-ins, and browser extensions. Vertical applications tackling a specific problem have the benefit of being able to focus on fine-tuning models based on a focused data set (patient data for a healthcare use case). For startups operating in a specific vertical to succeed they must find a single problem/use case to solve and do it so well with easy and simple UX that users can no longer imagine their lives without it - once they start, they are hooked and can’t stop.
Success Criteria in AI Business Models
We see that startups are launching fast, but it is difficult to know for certain which startup is building a Generative AI Native product from ground-up as opposed to creating a new AI feature inside a traditional SaaS product. We also notice that many startups are building AI products for the same use cases (marketing, copywriting, writing) without enough product differentiations, making it hard to conclude which startup has a unique edge or moat, and what solutions could be useful to end customers and succeed. Most of the early startups are building products using similar, if not the same, models trained on the same data sets with the same architectures. Howser, AI startups could find their differentiation in some of the following ways:
Proprietary Data:
To differentiate in the age of AI, startups will need proprietary data sets derived from aggregate usage patterns and user interactions. Every additional data point fed to the AI model and the way the user reacts to model’s responses, make the foundational model smarter. Generally, the higher quality data that can be fed into the models, the more highly personalized and customized the product can serve its users. Apps solely built on other foundation models without a proprietary dataset, and whose ML models don’t get better with use case specific data may not be able to differentiate. Startups may need to fine tune and improve the chosen baseline model using the new domain specific or use case specific data or customer data (workflow data, health data, financial data) that is not accessible to competitors. AI with proprietary data creates sustainable value of the product for the end user. On the other hand, collecting large data may incur high investments in training infrastructure and sometimes regulatory friction. That’s why the startup must balance data acquisition and model training costs and incremental value of data and ensure that the data they are acquiring to refine the model results in a far superior better product.
User Experience:
User Experience (UX) is the style in which end users interact with a company’s products and services. UX could make or break a product as it could drive brand loyalty. Good UX makes it easy for the end user to accomplish their desired tasks in the most efficient and pleasant way possible. We notice a lot of copycat apps that are leveraging third-party foundation models and infrastructure and trying to distinguish solely via fluffy user experience. In the age of AI, the design choices could include considerations about the type and format of inputs/AI prompts and the type, format, and quality of outputs that users want in their specific context. For Generative apps, the end-user prompts the model, controls the workflow, and selects the best outputs. But the user doesn’t always know how best to interact with the model without some guidance. Most foundational models don’t have 100% accuracy. End user needs some help on how to interact with the AI model (prompt engineering) and negotiate the delusions to eventually obtain the correct output. So, the prompts the end user naturally inputs should be guided by an intuitive UX and understood/interpreted by the AI model to generate output that is expected and helpful to the user. Additionally, the product must integrate easily into user workflows from an easy prompt input and output perspective.
Ecosystem Advantage:
A startup’s potential to create its own ecosystem with multiple product offerings can be crucial in the new AI era. The product should operate in a way that easily works with existing products used by customers if they must see the RoI of AI in their work streams. A few strategic approaches for AI startups to consider are their products ability to seamlessly integrate with the already present software in customers environments, be a part of their current workflow and ability to create a suite of AI products that solves multiple problems in key day-to-day tasks and use cases. Building plugins into Microsoft, G-Suite, Salesforce, Workday, ServiceNow products are examples of seamless integrations. A startup that plays with a comprehensive ecosystem with a wide range of integrations, plugins, or content while leveraging proprietary data is hard to beat. In this regard, we are also seeing applications that have already accumulated millions of users are finding easy ways to incorporate AI into those apps and workflows.
Early Mover’s Advantage:
OpenAI has proved this. Being a first mover in a specific vertical or use case and dominating the GTM that can create a strong brand recognition and familiarity when the AI field is evolving, along with a large user base could be an advantage. This advantage can discourage others from launching or result in a situation where the gap is so wide that competitors are unlikely to catch up. But the first mover’s advantage may be hard to keep up given the increase in accessibility to AI models and new datasets. Startups must think about how to continuously gather and refine new data for the best results.
Despite how energetic the IA field has become in the last 3-4 months, At Service Ventures, we still think that there is a limitation of using these large pre-trained AI models - the lack of domain knowledge of these models. Startups are building many applications on top of these models, and for most, their products are run by small engineering and product teams that depend on magics of these large AI models, rather than founders understanding the model’s secret sauce. Without access to the models and without understanding how these large AI models of various providers work, founders will have heavy dependency on these models to work effectively. Ultimately startups may need to develop their own proprietary system if these large models are product's primary feature and founders have no clue what and how the AI model produces the outputs. On the flip side, if the model becomes the main feature, competitors will always be around the corner building to reach feature parity or overtake with a better product. A moat creates the separation between startup’s business and competitors that allows it to scale. For now, we feel like balancing this act of having vs. not having own AI model is becoming more crucial to most generative AI startups.
/Service Ventures Team
Comments