Stability AI unveils Stable Video Diffusion

This week witnessed a flurry of activity among startups, each pushing the boundaries of what AI can achieve. While OpenAI grappled with internal upheaval, other players continued to march forward, unveiling groundbreaking innovations. In this blog, we’ll delve into the latest developments from Stability AI, Meta, and Runway, exploring the potential and pitfalls of their cutting-edge AI models.

Amidst the chaos at OpenAI, Stability AI seized the spotlight by introducing Stable Video Diffusion. This AI model, an extension of Stability’s Stable Diffusion text-to-image model, takes a significant step into the realm of generative videos. In a research preview, Stability unveiled SVD and SVD-XT, models capable of transforming still images into high-quality videos, offering applications in education, creativity, and artistic processes.

However, concerns linger about potential misuse, especially in the absence of a built-in content filter. Given past incidents with similar AI models, there’s a cautious eye on the terms of use and the model’s future circulation, with implications for ethical use and legal considerations around copyright. Despite limitations, Stability AI remains optimistic about the extensibility of their models, hinting at future applications like 360-degree views of objects. With a recent $25 million funding injection, Stability AI aims to commercialize Stable Video Diffusion, eyeing applications in advertising, education, entertainment, and beyond.

The genesis of Stable Video Diffusion lies in the models — SVD and SVD-XT — both trained on a dataset of millions of videos and fine-tuned on a smaller set of hundreds of thousands to around a million clips. While the source of the training data remains unclear, Stability’s commitment to transparency is evident in its whitepaper and communication regarding the intended and non-intended applications of their video-generating models.

Meta, formerly Facebook, entered the AI language model arena with LLaMA. Unlike its counterparts ChatGPT and Bing, LLaMA was released as an open-source package, inviting researchers to explore and democratize AI access. However, just a week after its debut, LLaMA faced a leak, stirring debates on responsible sharing of cutting-edge research.

The leak sparked discussions on the potential consequences, with some foreseeing personalized spam and phishing attempts. Others defended open access, emphasizing the importance of widespread testing to identify vulnerabilities and enhance robustness. The leaked model’s power, especially its ability to run on a single A100 GPU, adds a new dimension to the AI research landscape.

LLaMA’s leak also ignited the ongoing debate between proponents of open and closed systems in the AI community. While some argue for greater access to AI research and models, others caution against the potential dangers of a free-for-all approach. LLaMA’s inadvertent release into the public domain further fuels this ideological struggle, highlighting the challenges of striking a balance between openness and cautious dissemination.

In the dynamic world of AI image generators, Runway faced a dilemma with its Stable Diffusion XL v0.9 leaking online before an official release. The new model promises ultra-photorealistic imagery, competing with counterparts like MidJourney, but its clandestine spread raises questions about control and security.

Runway’s co-founder, Anastasis Germanidis, acknowledged the challenges of training AI models on artists’ works while navigating copyright concerns. The generative AI landscape has seen legal battles, with artists suing companies for alleged copyright infringement. To address this, some AI vendors, including Stability AI, have introduced opt-out mechanisms for artists and communal funds to share revenue with contributors.

While Runway is yet to implement such measures, Germanidis hinted at the company’s commitment to working with artists and exploring options to ensure ethical AI use. As copyright debates intensify, the leaked model puts Runway in a challenging position, prompting discussions on responsible AI development and legal safeguards.

The leaked models, whether for video generation, language processing, or image creation, reveal the dual nature of AI’s potential: a tool for innovation and creativity, but also a potential source of ethical and legal quandaries. As these technologies progress, the industry must navigate uncharted waters, fostering collaboration between AI developers, artists, and legal experts to ensure a responsible and ethical future for artificial intelligence.

The leaked models, such as Stable Video Diffusion, LLaMA, and Stable Diffusion XL v0.9, expose not only the capabilities of AI but also the legal and ethical challenges that accompany them. Generative AI has brought copyright issues to the forefront, with artists and authors raising concerns about the use of their works in training these models.

This week, authors, including George R.R. Martin, filed a lawsuit against OpenAI, alleging that ChatGPT was trained on their work without consent. Similar concerns echo in the generative image generation space, with artists suing Stability AI, MidJourney, and DeviantArt for alleged copyright infringement.

Onstage at Disrupt 2023, Anastasis Germanidis from Runway highlighted the company’s ongoing exploration of the right approach to training AI models on artists’ works. As the industry grapples with these challenges, some AI vendors have introduced mechanisms for artists to opt out of model training and communal funds to share revenue.

However, the debate extends beyond protecting artists’ rights to the question of whether AI-generated works can be copyrighted. The U.S. Copyright Office has recently sought public input on generative AI and IP, indicating the complexity of the issue. Runway’s assertion that generated content can be copyrighted adds another layer to this evolving legal landscape.

As AI continues to push boundaries, the leaked models and legal battles underscore the need for a delicate balance between innovation and responsibility. The AI landscape is evolving rapidly, with startups racing to introduce groundbreaking models while grappling with ethical considerations and legal challenges.

The leaked models, whether for video generation, language processing, or image creation, reveal the dual nature of AI’s potential: a tool for innovation and creativity, but also a potential source of ethical and legal quandaries. As these technologies progress, the industry must navigate uncharted waters, fostering collaboration between AI developers, artists, and legal experts to ensure a responsible and ethical future for artificial intelligence.

In a world where AI is both a creator and a potential copyright violator, the journey forward demands a thoughtful and collective approach. As the AI saga unfolds, we can only anticipate more revelations, challenges, and breakthroughs in this dynamic and ever-expanding field, surpassing our exploration of the present and future of artificial intelligence. For all my daily news and tips on AI, Emerging technologies at the intersection of humans, just sign up for my FREE newsletter at www.robotpigeon.beehiiv.com