Rephrase.AI : Reinventing Video Creation

Intro:

If the ideas in the caption sound futuristic, deeply intriguing, and cool, we assure you; we felt the same flurry of excitement while having a conversation with Ashray Malhotra, CEO, and co-founder of Rephrase.AI. 

Rephrase.AI is pioneering a new type of storytelling with a powerful video creation tool for effective and affordable synthetic media creation. A Deep-Tech startup (business model based on high tech innovation in engineering), backed by Lightspeed Venture Partners and AV8 Ventures ($1.5 Million seed funding), works with B2B, SaaS businesses, brands, and agencies to create personalized sales, marketing, and corporate training content. 

It has engineered facial reanimation technology with the help of Computer Vision scientists and a team of incredibly dedicated AI researchers from around the globe. All founders have recently featured in the Forbes 30 under 30 Asia 2021 cohort. 

Product Hunt is a community where founders, product enthusiasts, and geeks go to check out the best new products and get attention for tech products that they’ve built. If you launch successfully on Product Hunt, you can quickly grow your subscribers and customers. Rephrase.AI had also launched its groundbreaking product on Product Hunt earlier this year, and stood in the Top 3 products for an entire week!

A glance at Rephrase’s SaaS dashboard Tutorial (https://www.rephrase.ai/tutorial) will draw you in and leave you with a sense of amazement. We highly recommend you check it out.

Co-founders:

Ashray Malhotra (ex-Goldman Sachs and MIT media Labs, Elec IITB ‘15), Shivam Mangla (ex-Facebook, CSE IITR ‘15), and Nisheeth Lahoti (ex-Google, Samsung, CSE IITB ‘15) founded Rephrase.AI in Bengaluru, 2018. It is one of the top 10 startups exclusively selected for the Techstars India Accelerator program and has been featured in the prestigious Tech30 of Yourstory.

Rephrase.AI is the second entrepreneurial venture for IITB batch mates Ashray and Nisheeth, SoundRex being their first startup that decreased latencies in audio transmission using ML algorithms. Shivam was a software engineer at Facebook’s Menlo Park headquarters before co-founding Rephrase with the other two.

How Rephrase.AI is empowering its users:

“The platform lets the user pick a model, choose a voice, background, lighting, and creates videos out of any text of choice (a simple script!) variables, and they can then upload a CSV file with name, email address, and other variables. We can then create millions of personalized videos with values of the merge tags uploaded from the CSV files,” says Ashray.

“It’s a video production cloud studio, with virtual cinematographers, camera people, and actors. You just tell them your script, and they do the rest.”

The product can help sales and marketing teams increase their conversions by 400% using personalized videos at scale & make their customers feel special.

Tech behind the scenes:

A sophisticated deep learning engine learns facial features corresponding to spoken audio (lip movements, facial expressions, and head movements) from existing video footage of a model speaking directly to a camera for 10 minutes. The startup’s ML algorithms (like Generative Adversarial Networks) reconstruct a lifelike video using predictive analytics to overlay any audio file on any video (automating everything from script to video).

For those of us who are not AI engineers: Their ML algorithms correlate speech with the movement of facial features. When someone types in a script, a text-to-speech engine converts it to audio, and their ML engine generates a lifelike video of a model speaking that text (using what it learned earlier).

A simple API call or a web app can access this engine.

What planted the seed?:

“Nisheeth mentioned to me he wanted to build an engine which can take text as input and create a Hollywood level movie as output so that we can just take a script and create a movie without ever shooting anything,” said Ashray in an interview with Inc42.

Having had previous exposure to filmmaking as an ex-Silver Screen Convener in his sophomore year, Ashray quickly realized the quantum of time, effort, and money this idea might save.

While making a Hollywood quality movie from script remains the futuristic ambition of the startup, until the technology advances sufficiently, Rephrase.AI developed a monetizable tech product to automate video creation that businesses can use.

The Business Problem to be solved:

The traditional video creation pipeline is slow, and resource-intensive, meaning that it cannot be scaled economically.

“In general circumstances, what you would have to do is first conduct model auditions, then get a cinematographer, rent a studio, shoot and then do post-production shoot editing. We eliminated all of those tasks and made the entire process only a few minutes long.”

Using AI to circumvent the tedious procedure (hiring models, booking a studio, photographers, directors, and editors) provides a more straightforward and scalable approach, which can generate millions of personalized videos with just a  few clicks.

“Now, enterprises can create high-quality video content for targeted sales and marketing initiatives while saving time and money. This technology will change the way we think about video production, both for business communication today and filmmaking in the future.”

 “Now, instead of something which used to cost a hundred thousand dollars and requires dedicated VFX artists, we’re able to make it affordable for everyday use cases across a bunch of other different industries and across a bunch of different sized companies,” said Malhotra. 

Use cases:

An entire universe of possibilities opens up with such a powerful video creation tool. As a first step, the startup is focussing on:

• Corporate Communication: Transforming boring corporate messages and blogs to videos.
• Educational Content Creation: Simplifying Ed-Tech from studio-based recording to generating video tutorials just from scripts
• Learning and Development: Internal training and other corporate applications, say, for compliance.
• Marketing Content Creation: Creating multiple versions of an advertisement and brand videos targeting different consumer segments.
• Personalized Videos with Celebs: Record a celeb for say 10 mins, and use it to generate unlimited personalized content for branding, sales, and advertising.

Clientele:

Rephrase has worked with Amazon, Bajaj Finserv, and Lowe’s Innovation Labs, with more to come in the future.

Competitors:

The startup faces immense competition from London-based Synthesia.io, which is working on similar technology. It has raised  $4 Million in funding and has worked with big clients like Reuters, WPP, McCann, BBC, and Accenture. 

The startup also faces indirect competition from Video-powered startups like Rocketium, Invideo, and others like Vidyard, Loom, Wochit, Animoto, and Wibbitz. They automate video creation but do not use AI to generate videos of human models.

Ethics of synthesized media:

The usual discussions about morality and authenticity are critical for synthesized videos, especially when it involves manipulating human faces and the spoken word. In the wrong hands, this tool can proliferate false news, propaganda, identity theft, and much more. 

However, Rephrase is committed to setting and meeting a high bar, with its firm ethics policy resting on the pillars of responsible access, transparency, and inclusion. It plans on allowing access to only the most trustworthy and verified clients (organizations and institutions) and keeping the platform closed to individuals.

Road to the future:

While pooling in revenues from its current product offerings, the company plans to progressively enhance the technology by creating multiple emotions, hand gestures, head movements along with different camera angles.

“With each of these small tech jumps, you’re going to see us get a step closer to the virtual movie idea,”

We congratulate the co-founders for getting featured in Forbes 30 under 30 and wish that the pioneering team continues to innovate.

0