Here’s What SGE Can Do

Google is rolling out its new Search Generative Experience, which allows users to generate images with AI. Find out what else it can do.

Google is bringing generative artificial intelligence to its Search platform to offer users new ways of finding, visualizing and creating content.

The company began rolling out its generative AI-powered Search Generative Experience to select U.S. users on October 12, saying in a blog post that the update was the culmination of the work it has been doing over the years to make its search engine smarter and more intuitive.

Jump to:

Table of Contents

What is Google SGE, and how does it work?

SGE allows Google users to generate AI images and text by typing a prompt into the Google Search bar, working much in the same way as AI-powered text-to-image generators like Midjourney and DALL-E 2 and acting as a rival to Microsoft’s GPT-4 powered Bing Chat.

Google said SGE could help in situations where users might be looking for a certain image they can’t find in traditional search results, or when they need help visualizing an idea.

Users who opt in to SGE will see an option to create AI-generated images directly in Google Images as part of the model’s testing period. This feature is designed to appear when users are searching for inspiration, said Google (e.g., “minimalist Halloween table settings” or “spooky dog house ideas.” Users can then fine-tune the results by modifying the description.

“Tap on any of those images and you’ll see how generative AI has expanded your initial query with descriptive details, like ‘a photorealistic image of a capybara wearing a chef’s hat and cooking breakfast in a forest, grilling bacon’,” explained Google.

“From there, you can edit the description further to add even more detail and bring your vision to life. Maybe you want to see the capybara chef making hash browns instead, or you want to add a light blue background with clouds.”

The image generation capability offered by SGE is only available in the U.S. to people who opted into the trial phase and are aged 18 or older.

SEE: Artificial Intelligence: Cheat Sheet (TechRepublic)

SGE builds on a number of major AI advancements pioneered by Google in recent years, including its BERT natural language processing engine and even more powerful Multitask Unified Model — or MUM — a large language model reported to be 1,000 times more powerful than BERT.

According to Google, SGE is powered by a variety of LLMs, including an advanced version of MUM and PaLM2, which took center stage at Google I/O 2023.

Google has made a concerted effort to lean into growing consumer interest around AI, putting the technology at the center of the marketing around its flagship Google Pixel 8.

Google SGE can help users create written content

Similar to the capabilities offered by Google Bard and rival platform ChatGPT, SGE will also help users generate written content (Figure A).

Figure A

SGE can help users with written tasks, too. Image: Google

Google gave the example of someone looking for home improvement advice. “After finding helpful ideas across the web and contractors you’d like to get in touch with, you can ask SGE to ‘Write a note to a contractor asking for a quote to turn my garage into a home office’,” the company said.

SGE will draft a note based on the prompt, which users can revise for length, tone or language as needed. These can then be exported to Google Docs or Gmail.

SEE: Hiring kit: Prompt engineer (TechRepublic Premium)

Google is building in content safeguards

Google said it was conscious of safety concerns around the use of generative AI and AI-generated content; and to that end, it was building in safeguards to block images that go against its prohibited use policy. This includes any content considered harmful or misleading.

Additionally, all images generated by SGE will be watermarked and contain metadata indicating that they have been AI-generated, while another upcoming Google tool called “About this image” will allow users to better ascertain the “context and credibility of images.”

Google explained: “For example, it might show you when a similar version of this image may have first been seen by Google; or show you other pages on the web that use a similar image, including news or fact checking sites.”

SEE: Google AI in Workspace Adds New Zero-Trust and Digital Sovereignty Controls (TechRepublic)

Future uses for SGE

Further ahead, Google hopes SGE will help people in situations where various contextual or situational information is needed — for example, while shopping or searching local services. In these scenarios, SGE will pull from Google’s Shopping Graph to provide product information like reviews, prices and images or relevant information about local places to help consumers compare options.

SGE will also be used to serve ads in search results, as disclosed in Google’s documentation on SGE.

Exactly what role generative AI will play here isn’t clear, but it does suggest that any ads will be clearly labeled so they are distinguishable from organic results. “As Search applies the power of generative AI, Search ads will continue to play a critical role,” Google said.

The company continued: “In this new experience, advertisers will continue to have the opportunity to reach potential customers along their search journeys. We’ll continue to test and evolve the ads experience as we learn more. As always, we’re committed to transparency and making ads distinguishable from organic search results.”