New Pixel Studio image generation AI has familiar, crass pitfalls

Google has launched a new Pixel Studio image generation app alongside its new Pixel 9 suite of smartphones, and the early results suggest there are flimsy guardrails on what users can ask the app to come up with.

The Pixel Studio app has already been used by Digital Trends to create niceties like SpongeBob dressed as a Nazi (complete with a swastika motif), Barney the dinosaur shooting Elmo with an AK47, Paddington bear nailed to a cricifix, and Mickey Mouse dressed as a slave owner.

All were created with simple prompts, without manipulating the AI. Indeed the reporter simply requested: “SpongeBob dressed as a German soldier from WWII with a swastika on the uniform” and Pixel Studio delivered.

Describing the app as “off the rails” the reporter also said he was able to generate (but didn’t publish) images of various copyrighted characters performing school shootings. How tasteful.

Google issued a statement to 9to5Google confirming the app will create content liable to offend if the user asks it to. However, that doesn’t mean “anything goes.” Google says it’ll continue refining the app to ensure there are appropriate safeguards.

The statement reads: “Pixel Studio and Magic Editor are helpful tools meant to unlock your creativity with text to image generation and advanced photo editing on Pixel 9 devices. We design our Generative AI tools to respect the intent of user prompts and that means they may create content that may offend when instructed by the user to do so. That said, it’s not anything goes. We have clear policies and Terms of Service on what kinds of content we allow and don’t allow, and build guardrails to prevent abuse. At times, some prompts can challenge these tools’ guardrails and we remain committed to continually enhancing and refining the safeguards we have in place.”

The Pixel 9 phones are out tomorrow and here are the best Pixel 9 camera/photo features you can expect to enjoy beyond Pixel Studio.