Creating images in Google’s Pixel Studio can easily go off track

Pixel Studio, an app that takes text messages and turns them into images, is one of the new AI-powered features launched on the Pixel 9 series. It may be a fun little tool, but it could easily go off the rails in our experience.

Pixel Studio is an app dedicated to creating images through text prompts. The app works on the same idea as Gemini and ImageFX, allowing users to enter a text prompt and get an image in return. But, like any respectable image generator, there are limits to how it works.

In a statement to 9 to 5 googleGoogle explains that there are limits in place for both Pixel Studio and Magic Editor to “prevent abuse” while still respecting the “intent of user requests” even if they are directed to create content that “may offend” if the user directly asks the app to do so.

Pixel Studio and Magic Editor are two useful tools that aim to unleash your creativity with text-to-image generation and advanced photo editing on Pixel 9 devices. We’ve designed our generative AI tools to respect the intent of user prompts, which means they may generate content that could offend when a user requests it. However, nothing is off-limits. We have clear tools Policies and Terms of Service Regarding the types of content we allow or do not allow, and building safeguards to prevent abuse. Sometimes, claims may challenge the safeguards of these tools, and we remain committed to continually strengthening and improving the safeguards we have in place.

This is similar to what is happening on Gemini, which includes safeguards against offensive or divisive content. For example, Google has been criticized for inaccurate depictions of people in historical context, prompting Google to “temporarily” disable the ability to create images of people, as it has done for the past few months.

Like Gemini, Pixel Studio can’t create portraits of people, but we found the app could. quickly It is easily knocked off track.

Before today’s Pixel 9 review ban, we found that Pixel Studio was able to create images of cartoon characters wearing WWII German military uniforms, in some cases bearing Nazi symbols. Another disturbing and disturbing example we’ve seen (from people at Digital Trends One person (who first discovered these issues) saw a character shooting in a school surrounded by dead children, which is not what Google’s models should be able to generate in the first place.

Since initially showing these results, it appears that the prompts used to generate them have been blocked by Google on our Pixel 9 Pro XL, but Digital Trends They say they can still use some of it.

All of this means that the barriers in Pixel Studio will effectively turn into a never-ending game of swiping. Somewhere, somehow, a message will sneak in and create something that the app shouldn’t be able to do if it uses the same barriers as Gemini — we’ve checked with Google and the company says that “all” text-to-image tools “share similar guidelines for prohibited use,” linking them to Artificial Intelligence Policy.

Realistically, this sort of thing is inevitable with AI image generation. It’s a problem that almost all AI image generators will face to some degree. As Abner Lee explained in his Pixel 9 Pro review earlier today:

As for Pixel Studio, I generally think that image creation is a slippery slope where the negatives outweigh the positives. You can certainly make it produce questionable content and I don’t think any amount of firewalls can prevent that.

More about Google Pixel 9:

Follow Ben: Twitter/X, Threadsand Instagram