Archives

Emerging-Technologies

Hypotenuse AI wants to take the strain out of copywriting for ecommerce

Imagine buying a dress online because a piece of code sold you on its ‘flattering, feminine flair’ — or convinced you ‘romantic floral details’ would outline your figure with ‘timeless style’. The very same day your friend buy the same dress from the same website but she’s sold on a description of ‘vibrant tones’, ‘fresh cotton feel’ and ‘statement sleeves’.

This is not a detail from a sci-fi short story but the reality and big picture vision of Hypotenuse AI, a YC-backed startup that’s using computer vision and machine learning to automate product descriptions for ecommerce.

One of the two product descriptions shown below is written by a human copywriter. The other flowed from the virtual pen of the startup’s AI, per an example on its website.

Can you guess which is which?* And if you think you can — well, does it matter?

Screengrab: Hypotenuse AI’s website

Discussing his startup on the phone from Singapore, Hypotenuse AI’s founder Joshua Wong tells us he came up with the idea to use AI to automate copywriting after helping a friend set up a website selling vegan soap.

“It took forever to write effective copy. We were extremely frustrated with the process when all we wanted to do was to sell products,” he explains. “But we knew how much description and copy affect conversions and SEO so we couldn’t abandon it.”

Wong had been working for Amazon, as an applied machine learning scientist for its Alexa AI assistant. So he had the technical smarts to tackle the problem himself. “I decided to use my background in machine learning to kind of automate this process. And I wanted to make sure I could help other ecommerce stores do the same as well,” he says, going on to leave his job at Amazon in June to go full time on Hypotenuse.

The core tech here — computer vision and natural language generation — is extremely cutting edge, per Wong.

“What the technology looks like in the backend is that a lot of it is proprietary,” he says. “We use computer vision to understand product images really well. And we use this together with any metadata that the product already has to generate a very ‘human fluent’ type of description. We can do this really quickly — we can generate thousands of them within seconds.”

“A lot of the work went into making sure we had machine learning models or neural network models that could speak very fluently in a very human-like manner. For that we have models that have kind of learnt how to understand and to write English really, really well. They’ve been trained on the Internet and all over the web so they understand language very well. “Then we combine that together with our vision models so that we can generate very fluent description,” he adds.

Image credit: Hypotenuse

Wong says the startup is building its own proprietary data-set to further help with training language models — with the aim of being able to generate something that’s “very specific to the image” but also “specific to the company’s brand and writing style” so the output can be hyper tailored to the customer’s needs.

“We also have defaults of style — if they want text to be more narrative, or poetic, or luxurious —  but the more interesting one is when companies want it to be tailored to their own type of branding of writing and style,” he adds. “They usually provide us with some examples of descriptions that they already have… and we used that and get our models to learn that type of language so it can write in that manner.”

What Hypotenuse’s AI is able to do — generate thousands of specifically detailed, appropriately styled product descriptions within “seconds” — has only been possible in very recent years, per Wong. Though he won’t be drawn into laying out more architectural details, beyond saying the tech is “completely neural network-based, natural language generation model”.

“The product descriptions that we are doing now — the techniques, the data and the way that we’re doing it — these techniques were not around just like over a year ago,” he claims. “A lot of the companies that tried to do this over a year ago always used pre-written templates. Because, back then, when we tried to use neural network models or purely machine learning models they can go off course very quickly or they’re not very good at producing language which is almost indistinguishable from human.

“Whereas now… we see that people cannot even tell which was written by AI and which by human. And that wouldn’t have been the case a year ago.”

(See the above example again. Is A or B the robotic pen? The Answer is at the foot of this post)

Asked about competitors, Wong again draws a distinction between Hypotenuse’s ‘pure’ machine learning approach and others who relied on using templates “to tackle this problem of copywriting or product descriptions”.

“They’ve always used some form of templates or just joining together synonyms. And the problem is it’s still very tedious to write templates. It makes the descriptions sound very unnatural or repetitive. And instead of helping conversions that actually hurts conversions and SEO,” he argues. “Whereas for us we use a completely machine learning based model which has learnt how to understand language and produce text very fluently, to a human level.”

There are now some pretty high profile applications of AI that enable you to generate similar text to your input data — but Wong contends they’re just not specific enough for a copywriting business purpose to represent a competitive threat to what he’s building with Hypotenuse.

“A lot of these are still very generalized,” he argues. “They’re really great at doing a lot of things okay but for copywriting it’s actually quite a nuanced space in that people want very specific things — it has to be specific to the brand, it has to be specific to the style of writing. Otherwise it doesn’t make sense. It hurts conversions. It hurts SEO. So… we don’t worry much about competitors. We spent a lot of time and research into getting these nuances and details right so we’re able to produce things that are exactly what customers want.”

So what types of products doesn’t Hypotenuse’s AI work well for? Wong says it’s a bit less relevant for certain product categories — such as electronics. This is because the marketing focus there is on specs, rather than trying to evoke a mood or feeling to seal a sale. Beyond that he argues the tool has broad relevance for ecommerce. “What we’re targeting it more at is things like furniture, things like fashion, apparel, things where you want to create a feeling in a user so they are convinced of why this product can help them,” he adds.

The startup’s SaaS offering as it is now — targeted at automating product description for ecommerce sites and for copywriting shops — is actually a reconfiguration itself.

The initial idea was to build a “digital personal shopper” to personalize the ecommerce experence. But the team realized they were getting ahead of themselves. “We only started focusing on this two weeks ago — but we’ve already started working with a number of ecommerce companies as well as piloting with a few copywriting companies,” says Wong, discussing this initial pivot.

Building a digital personal shopper is still on the roadmap but he says they realized that a subset of creating all the necessary AI/CV components for the more complex ‘digital shopper’ proposition was solving the copywriting issue. Hence dialling back to focus in on that.

“We realized that this alone was really such a huge pain-point that we really just wanted to focus on it and make sure we solve it really well for our customers,” he adds.

For early adopter customers the process right now involves a little light onboarding — typically a call to chat through their workflow is like and writing style so Hypotenuse can prep its models. Wong says the training process then takes “a few days”. After which they plug in to it as software as a service.

Customers upload product images to Hypotenuse’s platform or send metadata of existing products — getting corresponding descriptions back for download. The plan is to offer a more polished pipeline process for this in the future — such as by integrating with ecommerce platforms like Shopify .

Given the chaotic sprawl of Amazon’s marketplace, where product descriptions can vary wildly from extensively detailed screeds to the hyper sparse and/or cryptic, there could be a sizeable opportunity to sell automated product descriptions back to Wong’s former employer. And maybe even bag some strategic investment before then…  However Wong won’t be drawn on whether or not Hypotenuse is fundraising right now.

On the possibility of bagging Amazon as a future customer he’ll only say “potentially in the long run that’s possible”.

Joshua Wong (Photo credit: Hypotenuse AI)

The more immediate priorities for the startup are expanding the range of copywriting its AI can offer — to include additional formats such as advertising copy and even some ‘listicle’ style blog posts which can stand in as content marketing (unsophisticated stuff, along the lines of ’10 things you can do at the beach’, per Wong, or ’10 great dresses for summer’ etc).

“Even as we want to go into blog posts we’re still completely focused on the ecommerce space,” he adds. “We won’t go out to news articles or anything like that. We think that that is still something that cannot be fully automated yet.”

Looking further ahead he dangles the possibility of the AI enabling infinitely customizable marketing copy — meaning a website could parse a visitor’s data footprint and generate dynamic product descriptions intended to appeal to that particular individual.

Crunch enough user data and maybe it could spot that a site visitor has a preference for vivid colors and like to wear large hats — ergo, it could dial up relevant elements in product descriptions to better mesh with that person’s tastes.

“We want to make the whole process of starting an ecommerce website super simple. So it’s not just copywriting as well — but all the difference aspects of it,” Wong goes on. “The key thing is we want to go towards personalization. Right now ecommerce customers are all seeing the same standard written content. One of the challenges there it’s hard because humans are writing it right now and you can only produce one type of copy — and if you want to test it for other kinds of users you need to write another one.

“Whereas for us if we can do this process really well, and we are automating it, we can produce thousands of different kinds of description and copy for a website and every customer could see something different.”

It’s a disruptive vision for ecommerce that is likely to either delight or terrify — depending on your view of current levels of platform personalization around content. That process can wrap users in particular bubbles of perspective — and some argue such filtering has impacted culture and politics by having a corrosive impact on the communal experiences and consensus which underpins the social contract. But the stakes with ecommerce copy aren’t likely to be so high.

Still, once marketing text/copy no longer has a unit-specific production cost attached to it — and assuming ecommerce sites have access to enough user data in order to program tailored product descriptions — there’s no real limit to the ways in which robotically generated words could be reconfigured in the pursuit of a quick sale.

“Even within a brand there is actually a factor we can tweak which is how creative our model is,” says Wong, when asked if there’s any risk of the robot’s copy ending up feeling formulaic. “Some of our brands have like 50 polo shirts and all of them are almost exactly the same, other than maybe slight differences in the color. We are able to produce very unique and very different types of descriptions for each of them when we cue up the creativity of our model.”

“In a way it’s sometimes even better than a human because humans tends to fall into very, very similar ways of writing. Whereas this — because it’s learnt so much language over the web — it has a much wider range of tones and types of language that it can run through,” he adds.

What about copywriting and ad creative jobs? Isn’t Hypotenuse taking an axe to the very copywriting agencies his startup is hoping to woo as customers? Not so, argues Wong. “At the end of the day there are still editors. The AI helps them get to 95% of the way there. It helps them spark creativity when you produce the description but that last step of making sure it is something that exactly the customer wants — that’s usually still a final editor check,” he says, advocating for the human in the AI loop. “It only helps to make things much faster for them. But we still make sure there’s that last step of a human checking before they send it off.”

“Seeing the way NLP [natural language processing] research has changed over the past few years it feels like we’re really at an inception point,” Wong adds. “One year ago a lot of the things that we are doing now was not even possible. And some of the things that we see are becoming possible today — we didn’t expect it for one or two years’ time. So I think it could be, within the next few years, where we have models that are not just able to write language very well but you can almost speak to it and give it some information and it can generate these things on the go.”

*Per Wong, Hypotenuse’s robot is responsible for generating description ‘A’. Full marks if you could spot the AI’s tonal pitfalls

AI is struggling to adjust to 2020

2020 has made every industry reimagine how to move forward in light of COVID-19: civil rights movements, an election year and countless other big news moments. On a human level, we’ve had to adjust to a new way of living. We’ve started to accept these changes and figure out how to live our lives under these new pandemic rules. While humans settle in, AI is struggling to keep up.

The issue with AI training in 2020 is that, all of a sudden, we’ve changed our social and cultural norms. The truths that we have taught these algorithms are often no longer actually true. With visual AI specifically, we’re asking it to immediately interpret the new way we live with updated context that it doesn’t have yet.

Algorithms are still adjusting to new visual queues and trying to understand how to accurately identify them. As visual AI catches up, we also need a renewed importance on routine updates in the AI training process so inaccurate training datasets and preexisting open-source models can be corrected.

Computer vision models are struggling to appropriately tag depictions of the new scenes or situations we find ourselves in during the COVID-19 era. Categories have shifted. For example, say there’s an image of a father working at home while his son is playing. AI is still categorizing it as “leisure” or “relaxation.” It is not identifying this as ‘”work” or “office,” despite the fact that working with your kids next to you is the very common reality for many families during this time.

Image Credits: Westend61/Getty Images

On a more technical level, we physically have different pixel depictions of our world. At Getty Images, we’ve been training AI to “see.” This means algorithms can identify images and categorize them based on the pixel makeup of that image and decide what it includes. Rapidly changing how we go about our daily lives means that we’re also shifting what a category or tag (such as “cleaning”) entails.

Think of it this way — cleaning may now include wiping down surfaces that already visually appear clean. Algorithms have been previously taught that to depict cleaning, there needs to be a mess. Now, this looks very different. Our systems have to be retrained to account for these redefined category parameters.

This relates on a smaller scale as well. Someone could be grabbing a door knob with a small wipe or cleaning their steering wheel while sitting in their car. What was once a trivial detail now holds importance as people try to stay safe. We need to catch these small nuances so it’s tagged appropriately. Then AI can start to understand our world in 2020 and produce accurate outputs.

Image Credits: Chee Gin Tan/Getty Images

Another issue for AI right now is that machine learning algorithms are still trying to understand how to identify and categorize faces with masks. Faces are being detected as solely the top half of the face, or as two faces — one with the mask and a second of only the eyes. This creates inconsistencies and inhibits accurate usage of face detection models.

One path forward is to retrain algorithms to perform better when given solely the top portion of the face (above the mask). The mask problem is similar to classic face detection challenges such as someone wearing sunglasses or detecting the face of someone in profile. Now masks are commonplace as well.

Image Credits: Rodger Shija/EyeEm/Getty Images

What this shows us is that computer vision models still have a long way to go before truly being able to “see” in our ever-evolving social landscape. The way to counter this is to build robust datasets. Then, we can train computer vision models to account for the myriad different ways a face may be obstructed or covered.

At this point, we’re expanding the parameters of what the algorithm sees as a face — be it a person wearing a mask at a grocery store, a nurse wearing a mask as part of their day-to-day job or a person covering their face for religious reasons.

As we create the content needed to build these robust datasets, we should be aware of potentially increased unintentional bias. While some bias will always exist within AI, we now see imbalanced datasets depicting our new normal. For example, we are seeing more images of white people wearing masks than other ethnicities.

This may be the result of strict stay-at-home orders where photographers have limited access to communities other than their own and are unable to diversify their subjects. It may be due to the ethnicity of the photographers choosing to shoot this subject matter. Or, due to the level of impact COVID-19 has had on different regions. Regardless of the reason, having this imbalance will lead to algorithms being able to more accurately detect a white person wearing a mask than any other race or ethnicity.

Data scientists and those who build products with models have an increased responsibility to check for the accuracy of models in light of shifts in social norms. Routine checks and updates to training data and models are key to ensuring quality and robustness of models — now more than ever. If outputs are inaccurate, data scientists can quickly identify them and course correct.

It’s also worth mentioning that our current way of living is here to stay for the foreseeable future. Because of this, we must be cautious about the open-source datasets we’re leveraging for training purposes. Datasets that can be altered, should. Open-source models that cannot be altered need to have a disclaimer so it’s clear what projects might be negatively impacted from the outdated training data.

Identifying the new context we’re asking the system to understand is the first step toward moving visual AI forward. Then we need more content. More depictions of the world around us — and the diverse perspectives of it. As we’re amassing this new content, take stock of new potential biases and ways to retrain existing open-source datasets. We all have to monitor for inconsistencies and inaccuracies. Persistence and dedication to retraining computer vision models is how we’ll bring AI into 2020.

Hear how three startups are approaching quantum computing differently at TC Disrupt 2020

Quantum computing is at an interesting point. It’s at the cusp of being mature enough to solve real problems. But like in the early days of personal computers, there are lots of different companies trying different approaches to solving the fundamental physics problems that underly the technology, all while another set of startups is looking ahead and thinking about how to integrate these machines with classical computers — and how to write software for them. At Disrupt 2020 on September 14-18, we will have a panel with D-Wave CEO Alan Baratz, Quantum Machines co-founder and CEO Itamar Sivan and IonQ president and CEO Peter Chapman. The leaders of these three companies are all approaching quantum computing from different angles, yet all with the same goal of making this novel technology mainstream.

D-Wave may just be the best-known quantum computing company thanks to an early start and smart marketing in its early days. Alan Baratz took over as CEO earlier this year after a few years as chief product officer and executive VP of R&D at the company. Under Baratz, D-Wave has continued to build out its technology — and especially its D-Wave quantum cloud service. Leap 2, the latest version of its efforts, launched earlier this year. D-Wave’s technology is also very different from that of many other efforts thanks to its focus on quantum annealing. That drew a lot of skepticism in its early days but it’s now a proven technology and the company is now advancing both its hardware and software platform.

Like Baratz, IonQ’s Peter Chapman isn’t a founder either. Instead, he was the engineering director for Amazon Prime before joining IonQ in 2019. Under his leadership, the company raised a $55 million funding round in late 2019, which the company extended by another $7 million last month. He is also continuing IonQ’s bet on its trapped ion technology, which makes it relatively easy to create qubits and which, the company argues, allows it to focus its efforts on controlling them. This approach also has the advantage that IonQ’s machines are able to run at room temperature, while many of its competitors have to cool their machines to as close to zero Kelvin as possible, which is an engineering challenge in itself, especially as these companies aim to miniaturized their quantum processors.

Quantum Machines plays in a slightly different part of the ecosystem from D-Wave and IonQ. The company, which recently raised $17.5 million in a Series A round, is building a quantum orchestration platform that combines novel custom hardware for controlling quantum processors — because once quantum machines reach a bit more maturity, a standard PC won’t be fast enough to control them — with a matching software platform and its own QUA language for programming quantum algorithms. Quantum Machines is Itamar Sivan’s first startup, which he launched with his co-founders after getting his Ph.D. in condensed matter and material physics at the Weizman Institute of Science.

Come to Disrupt 2020 and hear from these companies and others on September 14-18. Get a front-row seat with your Digital Pro Pass for just $245 or with a Digital Startup Alley Exhibitor Package for $445. Prices are increasing next week, so grab yours today to save up to $300.