The State of Creative Gen-AI in Fall 2024

Creative AI in 2024

This fall, creative AI has deeply integrated into artistic processes across various fields. Tools like RunwayML Gen-3 for video generation and UdIO for music production are empowering users to achieve remarkable results. AI has evolved from a novelty to an essential partner in the creative journey, enabling users to bring their visions to life with more efficiency and precision.

The real impact of AI lies in how users leverage it. AI tools are now used to create, experiment, iterate, and refine, making the creative process more dynamic and collaborative. This has fostered an era of artistic exploration where human and machine contributions blend seamlessly.

UdIO Negative Prompting

UdIO has added negative prompting, allowing users to specify what they do not want in their generated music, no more accidental Creed-style vocals when aiming for a post-grunge vibe. Negative prompting gives users greater control, ensuring that AI-generated music aligns perfectly with their intended mood and style.

Side note: I've noticed LLM use of the word "Echo" a lot, I had asked ChatGPT o1-Preview about it and basically got an answer that, LLMs are basically just echoing back the user input in a way, so the concept of "Echoing" and "Echoes" comes up alot because its inherit to its nature?

MidJourney Character Ref and Hedra Lip Sync

MidJourney's character likeness feature is one of the most notable advancements this season. Users can take an image of a character and generate likeness variations instantly. This feature is revolutionizing character design, allowing easy exploration of styles and emotions while maintaining consistency. It's perfect for storytellers needing quick iterations to bring their characters to life.

MidJourney's character likeness capability allows users to maintain a consistent visual identity across projects. This is crucial for multimedia storytelling, where visual continuity enhances narrative cohesion.

No more having five different versions of a granny within a short span for my space-themed short.

Combined with Hedra another AI tool for image to video lip sync, full dramas can be assembled between these two tools alone.

Flux Pro 1.1 and New Platforms

Flux Pro 1.1 has gained traction, especially with the emergence of new platforms like flux-ai.io and together.ai. This rise of new platforms diverges from what MidJourney is, a single private gatekeeper. Flux, on the other hand, is more open, allowing more platform control with greater automation possibilities than what MidJourney offers. Flux Pro's growing ecosystem in public and commercial domains signals a promising evolution, finding unique ways to serve specific creative needs.

The increasing number of platforms supporting Flux Pro shows rising interest in specialized AI tools that cater to unique requirements. This diversification provides users with a wider array of options to meet their creative needs.

Runway Gen-3 New Features

Runway Gen-3 just released turbo support for first and last frames, enhancing control over video generation. This ensures smooth transitions and cohesive storytelling, especially for looping content.

The extend feature allows users to take the last frame and use it for another 10 seconds of footage. This has proven handy for generating longer sequences while maintaining consistency, something previously done manually with a trusty Python script, often with unpredictable results.

The extend feature simplifies maintaining a consistent visual narrative across scenes, allowing users to focus on creativity rather than technical challenges. This advancement saves time and effort, making it easier to produce high-quality video sequences.

Pika AI 1.5 - Pika Effects

Pika AI 1.5 introduced an interesting batch of Pika Effects, including the "Cakify" effect, which adds a cake-like aesthetic to any image. Whether it's a pygmy hippo or a household item, this fun effect adds charm and surrealism, encouraging playful creativity. This trend towards whimsical and surreal content reflects a desire for playful experimentation in artistic expression.

Pika 1.5 occupies a unique space in the AI video generation landscape, offering a balance between creative output and specialized effects. While not as groundbreaking as Kling or as creatively free as Hailuo, it provides users with a rich set of customization options, including the distinctive Pika effects

Kling 1.5 - Lip Sync

Kling 1.5 impresses with highly accurate lip-syncing capabilities. Despite the expense, Kling 1.5's quality makes it ideal for high-quality content creation. The accuracy of Kling 1.5's lip-sync technology has improved AI character animation possibilities, providing a level of realism that brings characters to life in a convincing manner.

Kling 1.5 represents the pinnacle of creative AI video generation, offering a perfect blend of innovative output and customization options. It excels in producing stable, high-quality content that pushes the boundaries of AI-generated video.

Hailuo AI

Hailuo stands out as a powerhouse for generating creative and interesting video content. It's a new tool for users who prioritize unique and eye-catching output over fine-tuned control. While it lacks customization features and editing capabilities, its strength lies in producing videos that can captivate audiences with minimal user input. The unlimited option makes it attractive for high-volume projects where quantity and creativity are key, such as social media content creation or rapid prototyping of video ideas.

The second Canadian History Video I produced was a mix of Kling and Hailuo.

Conclusion

Fall 2024 marks a turning point where creative AI has moved from novelty to a core element of artistic expression. We are moving beyond asking if AI can create, to asking, What can we create together? Creative AI is no longer just a tool; it is a partner, a collaborator, and an enabler of new forms of artistic expression. The possibilities for what we can create together are endless and more exciting than ever before.