Hello Tuners,

From GPT Pro users finally getting their promised voice agent to waiting for what one more portable “AI Inference Running Machine,” I wonder when the iPhone finally gets AI. Well, never mind.

This week shows a calm composure following the massive open source releases we saw last week with Llama 3.1 and Mistral Large, and we should be seeing some fancy finetuned models coming soon.

• So Sam, Is It Really Her?

• I Hope Canva Sources Data Properly!

• NVIDIA Looking to Monopolize Data Generation After Training Hardware

• Hey “Friend”, Is The “Rabbit” Your Soul Animal?

So Sam, Is It Really Her?

OpenAI is rolling out a new advanced voice mode for ChatGPT to a select group of ChatGPT Plus subscribers. Initially showcased at the GPT-4o launch event in May, this feature drew attention for its resemblance to Scarlett Johansson's voice and was subsequently delayed due to safety concerns, and we know she is not shy of tackling a multi-billion dollar corporate. Demonstrations at the event highlighted the mode's enhanced capabilities, such as the ability to handle interruptions and adapt storytelling on the fly.

Originally scheduled for a late June release in alpha, the rollout was postponed by a month to meet OpenAI's stringent launch standards. The company focused on improving the model's ability to detect and refuse inappropriate content, involving over 100 external testers to ensure robustness. According to OpenAI spokesperson Taya Christianson, additional filters have been implemented to block requests for generating copyrighted audio, addressing both safety and legal considerations amid heightened scrutiny of OpenAI's safety policies.

I Hope Canva Sources Data Properly!

Canva, the popular creative design platform, has announced its acquisition of Leonardo AI, a prominent startup specializing in generative AI for art and image creation. While financial details remain undisclosed, the acquisition includes Leonardo’s entire workforce, who will join Canva’s AI team. Canva plans to integrate Leonardo’s “Phoenix” foundation model and proprietary technology into its offerings, while Leonardo will continue to develop its platform independently.

This move aligns with Canva’s strategy to enhance its visual AI tools and compete with industry giants like Adobe and Shutterstock. The acquisition follows Canva’s previous purchases of Kaleido, Flourish, Affinity, and others, reflecting its commitment to expanding its AI capabilities. Leonardo AI, known for its advanced GenAI platform initially aimed at game development, has broadened its focus to include marketing, advertising, and more. Notably, since last month, we have been seeing generative features for designs being rolled back with Figma leading the charge, and with Canva’s move, one must beg the question is there truly a good AI design tool?

NVIDIA Looking to Monopolize Data Generation After Training Hardware

NVIDIA has launched the Nemotron-4 340B, a suite of open models designed to generate synthetic data for training large language models (LLMs). This release aims to support developers across industries such as healthcare, finance, manufacturing, and retail. The high-quality training data produced by Nemotron-4 340B can significantly enhance the performance and accuracy of custom LLMs, addressing the challenge of accessing robust datasets that are often costly and hard to obtain.

The Nemotron-4 340B family includes base, instruct, and reward models that together create a pipeline for generating synthetic data. These models are optimized for use with NVIDIA NeMo, an open-source framework that facilitates end-to-end model training, and the NVIDIA TensorRT-LLM library for efficient inference. Developers can download Nemotron-4 340B from the NVIDIA NGC catalog and Hugging Face, with the option to fine-tune models using the Train on DGX Cloud service. Soon, the models will also be available as an NVIDIA NIM microservice, providing a standard API for deployment. This innovation offers a scalable, cost-effective solution for developers aiming to build powerful LLMs with high-quality synthetic data.

Hey “Friend”, Is The “Rabbit” Your Soul Animal?

Avi Schiffmann is making waves in the tech world with his latest creation, an AI companion aptly named "Friend." Friend is designed to offer constant companionship, providing support, validation, and even brainstorming assistance. This shift from task-oriented AI to companionship is a bold move, positioning Friend as an empathetic and intelligent partner rather than just a digital assistant.

The Friend device is still in its prototype phase, with plans to release the first 30,000 units by January at $99 each. It features a round glowing orb with a built-in microphone, designed to be worn or clipped onto clothing. By merging the companionship concepts seen in services like Character.AI and Replika with a portable device, Schiffmann hopes to foster deeper, more meaningful relationships between users and their AI friends. The core concept seems to be however somewhat lost on most users on wearable smart assistants given the failures of Rabbit R1 and Humane Pin.

LLM Of The Week

LazyLLM

Apple has unveiled LazyLLM, a groundbreaking approach to enhancing the efficiency of long-context large language models (LLMs). This innovative method introduces dynamic token pruning, which significantly accelerates the inference process while maintaining high accuracy. Specifically, LazyLLM can speed up the prefilling stage of a Llama 2 7B model by a factor of 2.34.

The core idea behind LazyLLM is to selectively compute key-value pairs (KV) only for tokens that are crucial for predicting the next token. This selective computation is applied during both the prefilling and decoding stages, optimizing the model's performance without compromising accuracy. According to Apple, "LazyLLM allows language models to dynamically select different subsets of tokens from the context in different generation steps, even though they might be pruned in previous steps." This dynamic selection process ensures that the model efficiently focuses on the most relevant tokens at each step, resulting in a more streamlined and effective inference process.

Weekly Research Spotlight 🔍

Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach

A recent study highlights a significant comparison between long-context large language models (LLMs) and Retrieval-Augmented Generation (RAG) systems, revealing that while long-context LLMs tend to outperform RAG in terms of average performance, RAG remains a more cost-effective solution. Specifically, the study found that on average, long-context LLMs surpassed RAG by 7.6% for Gemini-1.5-Pro, 13.1% for GPT-4O, and 3.6% for GPT-3.5-Turbo, with newer models like GPT-4O and Gemini-1.5-Pro demonstrating superior long-context understanding compared to older versions like GPT-3.5-Turbo.

To address the balance between performance and cost, the researchers proposed a novel approach called Self-Route. This method leverages self-reflection to dynamically route queries either to RAG or long-context LLMs, depending on the query's nature. The Self-Route strategy was shown to significantly reduce computational costs while maintaining performance levels comparable to long-context models. The study suggests that integrating both approaches can optimize efficiency and effectiveness, a perspective increasingly supported by experts advocating for combined methodologies. However, the absence of models like Claude and other custom LLMs trained for RAG in the analysis leaves some questions open about the generalizability of these findings across different LLM frameworks.

Best Prompt of the Week 🎨

Today's Goal: Try new things 🧪

Acting as a Skill Development Planner

Prompt: I want you to act as a skill development planner. You will create a structured daily plan specifically designed to help individuals upskill their design abilities and dedicate time to learning design software. You will identify a target client profile, develop key strategies and action plans, select the tools and resources for effective skill enhancement, and outline any additional activities needed to ensure consistent progress. My first suggestion request is: "I need help creating a daily activity plan for someone who wants to upskill their design skills and invest time in studying design software.”

This Week's Must-Read Gem 💎