top of page
  • Writer's pictureSean D. Emory

Rethinking AI Investing in 2024: Beyond Language, Towards Multimodality

building blocks to investing in ai comapnies

In an era where technology reshapes our world daily, we at Avory & Co. believe it's crucial to dig deeper as breakthrough moments approach. 

Generative AI definitely seems like such a moment.

To be clear though, AI isn't exactly new; technologists have been experimenting with various forms of AI for a while. Beyond generative AI and language models (LLMs), AI covers a wide range, such as computer vision, explainable AI (XAI), reinforcement learning, robotics, and natural language processing. I had to look this up but in 1950 Alan Turing's "Computing Machinery and Intelligence" paper laid the foundation for the field of AI and how computers could exhibit intelligent behavior. He was right.

With that said, as we enter 2024, we wanted to share our updated perspective on AI's potential impact on the world. Our views are informed by conversations with experts on our podcast "Inside Scoop" AI engineers, and leaders like the ex-head of AI at ServiceNow, product leads from Meta's Facebook Reality Labs, and conversations we held in quiet with various developers and founders. We will have a range of AI experts at our Avory Idea Day 2024 coming up in the next month. Sign up here to be notified when we set the date!

Avory idea day

First, we believe AI is a massive shift, touching all aspects of life.

From backend development to user experiences, AI is no longer confined to IBM Watson commercials; it's woven into diverse industries, from your social media feed to personalized marketing emails from Temu, to Netflix recommendations to trick you into another 7 hours of binge-watching, and even whether you see this post.

2023 was dominated by language and text-based AI, particularly large language models (LLMs) driving the hype.

These models are fascinating as they carry enhanced contextual understanding, making them good at conversation, content creation, and problem-solving. The ease of access to these platforms from Bing to ChatGPT has fueled consumer interest, driving curiosity and plenty of interesting experimentation (we just built our first multi-model AI "machine", more to come. These platforms aren't perfect today however I think there is no turning back. This technology is in the wild. Despite the excitement, we firmly believe that for generative AI to further embed itself into workflows, both in business and for consumers, it must extend beyond ChatGPT online. It requires sleek interfaces on the front end, and platforms should persist in developing APIs to seamlessly incorporate this technology on the backend. We are seeing this from the likes of Replicate AI, HuggingFace and others.

Here are just a few recent developments highlighting the accelerating pace of LLM integration:

  • APPLE RUMOR: The rumor is Apple's Siri plans to leverage generative AI at its June developer conference, promising more natural conversations and deeper user personalization. Time will tell. Apple has been horrible with Siri, but these models may change the game for them.

  • MICROSOFT: Their Copilot key integrates AI tools into your workflow. We do not know exactly the user flow, but I can imagine it incorporates speech and maybe prompts an AI to review whatever you are working on... They have not been great with consumer experiences, so I will hold back my thoughts until the launch.

microsoft copilot button

  • LEGAL: A legal tech startup called Robin AI secures $26 million to automate contract drafting and negotiations, showcasing AI's inroads into traditionally paper-driven sectors. AI is front and center, but the interface seems elegant.

  • DEVELOPERS: Replicate AI raises $40 million to democratize access to AI models like Stable Diffusion 2.0 and Llama 2, empowering several millions of developers. This is key as these platforms provide the tools for developers to build AI into their front and backend.

These are just a handful of recent announcements, all focusing on text-centric outputs. However, as we move into 2024 and beyond, we believe the next big step lies in multimodal AI.

It is All About Multimodal AI in 2024

While traditional models focus primarily on text data, multimodal AI integrates information from various modalities like text, images, and audio. This allows for a more comprehensive understanding of the input data, resembling a more human-like perception. Imagine an AI system that can not only read text instructions but also watch a demonstration video and listen to spoken explanations.

We saw Google's fake attempt at this at their conference of showing multiple inputs and various outputs, only to later find that these were recorded in different sessions. Regardless though, Google is working hard on this and they have the best engineers, run their own data centers, and have +$100B of cash!

google gemini fake video

Here are some exciting examples of where multimodal AI takes center stage:

1. Personalized Healthcare: Analyzing medical images, vital signs, and patient history together for quicker, more accurate diagnoses and treatment plans. I can imagine that electronic health record plays a critical role. Oracle buying Cerner seems way more interesting now.

 2. Enhanced Shopping Experiences: Combining visual product details with reviews and customer interaction data to suggest personalized recommendations and predict buying trends. Adobe has been working on this for a long time. Now they have FireFly AI. Salesforce Einstein may now actually feel more like Einstein.

3. Automated Content Moderation: Identifying and moderating harmful content on social media platforms by analyzing text, images, and audio together. Meta or Facebook is the only platform that shows reporting on things like child endangerment, and how their models proactively found 99% of instances.

4. Smarter Customer Service: Technicians receiving sound and imagery to diagnose problems, and chatbots analyzing text, tone, and facial expressions to offer personalized assistance.

5. Autonomous Vehicles: Self-driving cars rely on various sensors like cameras, radar, and LiDAR to navigate complex environments safely and intelligently. This seems like one of the breakthrough moments for automous.

Moving forward, and as investors, we look to see the key building blocks to successful investing within AI.

sean emory at avory and his building blocks to ai

 We are focusing on the following:

  1. Proprietary Models as Foundation Models Likely Commoditized Overtime: The speed at which models have been created suggests that foundation models are likely commoditized. Meaning, that only those with deep pockets, access to computing infrastructure, and a massive user base will survive the eventual cost curve downward. This does not suggest that there isn't value in investing in independent companies building foundation models, it is simply our belief that they me most at risk of price and therefore margin compression over time. Especially if reliant on 3rd party computing resources. Therefore, distinguishing between those simply adopting off-the-shelf models and those building or distinctively leveraging models. Like we saw verticalized software ie Veeva CRM for Healthcare, and Salesforce in Finance, we will see the same here.

  2. Powering (Compute, Storage, Chips, Software): Companies excelling in building integrated software and hardware capabilities to power AI solutions. We have stuck to this view, and believe that the power of hardware and software, while difficult will reign supreme. Hardware vendors are poised to undergo familiar cycles where storage and computing become linked to economic cycles, mirroring patterns observed in the past.

  3. Unique Data: Relying solely on unique datasets, rather than open internet data, provides stronger defensibility. The key to economic moats lies in owning data that differs from your competitors, creating more value than what they can offer. This differentiation is crucial for sustainable advantage. Take Zillow, for instance. While they have access to MLS data, similar to everyone else, what sets them apart is the access to three times the amount of user touchpoints compared to their closest competitor. This extensive user interaction data gives them a distinctive edge in the market.

  4. Data Organization: Companies with superior data organization practices can efficiently leverage their data for better outputs at a lower cost. We spoke with someone critical to Walmart and revealed the challenge of siloed datasets, including inventory and visitor traffic. Two opportunities arise from this situation: first, companies like Snowflake, Databricks, and Teradata can assist in consolidating data, and second, companies that undergo the organizational process can drive more efficient outcomes. New companies will emerge here.

  5. Governance: Ensuring robust governance mechanisms for AI data is paramount. We spoke with Debu Chatterjee from Konfer (here). It became clear to us that governance was critical but not front and center at the moment. This involves maintaining data integrity, compliance with regulations, and establishing ethical practices. Companies with strong governance frameworks position themselves for long-term success and trust.


For investors, the field of AI presents both challenges and opportunities. While foundation models might become commoditized, companies excelling in integrated hardware-software solutions, leveraging unique data, and employing robust data organization practices are poised for success. Companies that can combine all these factors are likely best positioned. Also and just as important, identifying businesses that prioritize strong governance and ethical practices will be crucial for navigating the evolving landscape of AI. We at Avory & Co. are excited to invest in companies shaping the future of AI, ensuring that their innovations benefit both individual investors and society as a whole.

About Avory & Co.

Investing where the world is headed. Avory is a high-conviction equity asset manager focused on Secular Growth and Transformation Stories. Allowing data to tell our stories. Avory invests in valuable companies led by exceptional teams. Avory & Co. is located in Miami, Florida offering SMA's for institutions and high-net-worth individuals.

📺 YouTube Investment Research:

🎙️ Inside Scoop Podcast: Apple Podcast

🌐Avory Website:

📝Avory Annual Investment Letter: Read

⚡Coming Soon⚡

Avory will be launching an Active ETF mimicking Our Premier + Strategy. Get Excited. Reach out to find out more.

Disclosure: Not a recommendation to invest in any companies mentioned. Investment involves risk.


bottom of page