What is Google's new AI, AudioPalm?

Jun 26, 2023 Ankita Baidya

New AI

[{"selector":"#anim-447aa6fa-59ad-4cdf-b69d-d1fa38e7cdd7 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(1.3329760362987986%, -22.93577672377123%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-9323581c-e17b-4f4c-b25a-e0ec261b178d","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-60f22130-2aa5-4b71-a345-3df7bc51abdc","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-090c46b5-73cb-448a-ae6c-f19fc02bd628","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] AudiopaLM is a large language model for voice production and comprehension. (Photo Credits: Reuters)

models

[{"selector":"#anim-98423713-e0e9-4a8b-a8f7-b6bbbb5fb61c [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(1.3481040800741146%, -22.93577672377123%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-ae8e9f5c-f178-4ea2-b179-8831e537875f","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ba42a964-cea4-44d5-add0-79d5041aaaef","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c7324b93-a8e8-4859-bb60-9a40f9b3dc4c","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Text-based and voice-based language models, PaLM-2 and AudioLM and AudioPaLM, respectively, are combined into a single multimodal architecture. (Photo Credits: Reuters)

uses

[{"selector":"#anim-b34bfd26-2d08-4d18-b804-dd496027dae3 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(-4.345833793766108%, -5.560191214489377%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-917bd7a9-bb00-4da2-a922-3c338abff4b6","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-18bb2004-504b-4613-bbc2-ed810563d798","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d807e8b5-5a3b-4425-ae16-7ca3143c489e","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] This multimodal architecture can process and generate both text and speech for use in speech recognition and speech-to-speech translation applications. (Photo Credits: Reuters)

Info

[{"selector":"#anim-5c22c619-fd6f-4b74-9d8b-942ece6454f3 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(12.24515922543907%, 21.963485144253795%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-787d62ad-8bd3-49e2-b07e-8e697c56f833","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3f239ba4-1f02-4833-9531-79d5414a0351","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-1a91e33d-5840-4ccd-bf38-d4fee17c1f08","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] The linguistic information found solely in large language models like PaLM-2 and AudioLM is passed down to AudioPaLM. (Photo Credits: Reuters)

other Info

[{"selector":"#anim-94c8d4e3-9efd-47d2-8755-e06b1777bf20 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(-12.044897010361508%, 21.251824999000036%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-915d466f-090b-4832-955b-352cd2f52ca0","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-6c7bb076-c8c3-4983-930d-ff712e397a30","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9bdb1b11-5319-45ad-bf41-8304958dadc9","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] The capacity to preserve paralinguistic information like speaker identification and tone is also passed down to AudiopaLM. (Photo Credits: Reuters)

model

[{"selector":"#anim-4fbcb05a-0c3c-44c1-9784-f24c47d421cf [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0.3591608197286889%, -22.93577672377123%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-4769ff92-4531-4f84-81a4-4a0493730edd","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-550a53d5-bf5a-46ec-99b0-2a3126b80aaf","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-104f0c98-109b-42a7-8f9d-ad661d4cb2fe","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] The model performs voice translation tasks substantially better and and it can execute zero-shot speech-to-text translation for numerous language. (Photo Credits: Reuters)

voices

[{"selector":"#anim-6eb1970b-6dfb-440e-94f0-3e8160ddfd07 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(13.478412835506937%, -9.904089138181206%) scale(1.4587155963302751)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-a6c2e5d8-1089-4303-b57c-6f5013a3fac7","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-f422079b-c61f-47ff-ab45-fe7a41cf55f9","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7e07d807-e991-4ddb-bac6-a8a90163228d","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] AudioPaLM shows how audio language models work by transferring voices between languages in response to a brief spoken prompt. (Photo Credits: Reuters)

Examples

[{"selector":"#anim-1d2b9667-4605-4517-aa65-99bc03db7280 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(6.440539587864162%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-154f5c9a-ef32-4136-8a4a-e8024bd6f827","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5dfee75c-85aa-43d8-90c1-a911fa5e59c0","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-dbbc05e5-9f99-4d48-8817-ed0d0ae6edea","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7ff20cad-cb5c-49e9-9d0f-7485fcaf523c","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4601704b-3891-4c42-aa75-2ff6c3cb8c55","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-0fffa392-0192-4236-9694-94fa82c3c1d2","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7117935a-e481-4a89-afe7-1be9b9bf7584","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-6e3c3b46-ca3b-4322-83ec-0a64467c8a77","keyframes":{"opacity":[0,1]},"delay":0,"duration":1500,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] Speech-to-speech translation and automatic speech recognition are examples from the AudioPaLM model. Follow FE Tech Bytes Twitter LinkedIn Instagram Facebook (Photo Credits: Reuters) Read More