[audio logo]

Speaker 1: OTAN, Outreach and Technical Assistance Network.

[video playback]

- Hello, [spanish]. I am Carmen your virtual TDLS conference assistant. And today we are going to explore how computer generated queries can be used as powerful tools to enhance English learning. Today we're going to talk about how computer generated queries can help you learn English faster and more efficiently. With these powerful tools you can search for specific words, phrases, or even whole sentences that you want to learn. And then practice them until you feel confident using them in your everyday conversations.

Whether you're just starting to learn English or you're an advanced learner, computer generated queries can be a game changer. They're fully customizable to your specific needs and can help you reach your language learning goals in no time.

And now I'm excited to introduce our amazing presenters Barbara Van Dyken from Grossmont Adult Education, and Matthew Sussman CEO of New Voice Learning. They'll be sharing their expertise on how computer generated queries can transform your English learning experience. So get ready to be inspired and learn a lot from these two amazing speakers. Thank you for joining me today, and let's get ready for an unforgettable conference experience.

[end playback]

Barbara Van Dyken: Well, thank you, Carmen. Carmen, she's an AI-- she's an assistant-- she's our AI assistant for the TDLS conference in here today. And she is completely-- she was created using an AI art generator.

And she's brought to life with stunning detail and realism with the help of ChatGPT and Meraki, which are two very powerful AI tools. Her voice was crafted to give her a rich and believable accent. And then with the assistance of D-ID, which is an AI video generator, everything was brought together to animate her and bring her to life.

So if you're interested in creating your own animation, animated avatars like Carmen, you can check out the article in my reference-- in my references by just clicking on the QR code there, and the name of the article is called Artificial Corner. It offers step by step instructions on how to build your own artificial intelligence avatars like Carmen. Thank you, Carmen, for that wonderful introduction.

And then also in this presentation, all of the art you're going to see in the presentation was generated by AI art generators. And the thing about this is that the more descriptive you are, the more the AI-- the art generator will give you something close to what you're looking for. It won't give you exactly, but based on your description, it will give you something very, very close.

And I really like this because now I can create images, still images and also animated images, for my students that maybe reflect them, that look like them, that reflect the experiences that they're having, and that also help me to teach a concept because I can describe the graphic that I'm looking for to the art generator, and it will generate it for me.

Speaker 2: Can you watch the QR code, please.

Barbara Van Dyken: Oh, the Q-- yeah, let's see. The QR code?

Speaker 2: Yeah, I can't get it.

Speaker 3: I love that you put in citations.

Barbara Van Dyken: Yeah, oh, good. I can't-- yeah, they taught me well in grad school.

Speaker 2: That's what you going to say [ INAUDIBLE ]

Speaker 3: That's true.

Barbara Van Dyken: All right. OK. So all right everybody got the QR code?

[interposing voices]

Barbara Van Dyken: No? OK. Well, there'll be more opportunities. I've got them all over the place.

Speaker 2: OK.

Barbara Van Dyken: OK. So--

[relaxing music]

Oops. This way. Nice music.

[video playback]

- Hello--

[end playback]

Barbara Van Dyken: Did it start? OK. All right--

Speaker 2: It will be fascinating.

Barbara Van Dyken: This presentation is about search engine queries, chatbot queries, and AI tools. I'm going to go through it pretty quickly. So we're going to take a look at your search engine. Can you do a search? How is everybody even searching to find things? In the last-- I don't know. How many years has it been now? What's your go to for finding out information?

Speaker 2: Google.

[interposing voices]

Barbara Van Dyken: Good. OK. So we're going to-- it's a different a search than using a chatbot which is a ChatGPT, which is taking over. It's getting all the press right now. We're going to look at artificial intelligence tools, and we're going to showcase the one that Matt does, which is FLOW Speak .

And then the core of my presentation is I'm going to show you search activities that I use-- utilize queries to aid in language acquisition. So they're search engine queries that along with Wh-questions to teach Wh-questions. And they'll be running these through a search engine and then to compare in ChatGPT.

OK. So what's the difference between a question and a query? Anybody know? Anybody have that answer, the difference? Yes, of course, I'm going to give you the answer. So the difference this question is something you ask a person, right? And a query is what you ask the computer. OK. So if a search engine-- you're fishing, aren't you? It's like a fishing expedition. And so we're going to teach our students how to fish, and how to fish efficiently.

So the search engines they do primarily search and retrieve, so they're going in and they're just-- you make a request, and it goes in and it pulls back the information that you've requested. But what we've noticed is that usually what we get back is mostly advertisements. You have to sort through a bunch of advertisements before you get to the thing that you want. Whereas in the ChatGPT when you put in your search, it talks to you. It says, oh, I've got just the answer for you. So we'll take a look at that.

So the search and retrieve the students can find what they're looking for using relevant keywords, and in this lesson combined with the Wh-queries. So and then integrating the AI tools into the ESL experience. ChatGPT is what we call a conversational agent. It uses artificial intelligence, algorithms to understand what a person is saying, and then it responds in a natural human like way.

And also another thing about this is that, it's-- this algorithm is being trained to get to know you and to get to know how to respond to you. So-- and that's freaky. I think we need to delve a little bit more into that before we throw the baby out with the bathwater.

Speaker 2: I don't like that.

Barbara Van Dyken: Yes.

Speaker 2: They did because they're accustomed to trained to be our assistant because we're using the ChatGPT using our account?

Barbara Van Dyken: No, no. What it's doing it's analyzing your language. It's analyzing what your queries. It's taking a look at-- this is-- this is really beyond my pay grade, but I'm trying my best. What-- well, I think it is-- and please, somebody correct me if I-- because I'm still learning this myself. --but these the algorithm is learning the language of a general human not just me and [ INAUDIBLE ]

Well, I don't know. That's a good question, and I would encourage you to pursue it further. And then tell me about it. We're getting into some really scary territory here with this stuff, but I'm here to tell you don't be afraid. We're in control of this, I hope.

So this is what the studies have told us. And learners that use these database queries to access language learning materials, they're more autonomous in their learning, and they're more likely to use a range of resources to support their language acquisition

Speaker 4: Did you know there was a [ INAUDIBLE ]

Barbara Van Dyken: Yes. I've been trying to do that. Let's see. What was that? Control Shift and all--

[interposing voices]

Speaker 4: No, you have to make it disappear.

Barbara Van Dyken: OK. Good. Everything good?

Speaker 2: Yeah.

Barbara Van Dyken: All right. OK. Doing these queries they enable students to access authentic texts, and it results in an improvement in reading comprehension. And then Kim and Han found this that compared to their control group that this is that the ones that were using the querying databases were improving their reading comprehension.

OK. All right. And then Xu and Wang revealed that learners who use queries to look up the meaning of unfamiliar words showed a significant improvement in vocabulary acquisition compared to their control group. Not used to these big board. Sorry those of you online who are a little impatient. If you are, maybe you're having coffee.

So AI has a promising future in English language teaching with positive results in areas such as language skills, translation, and assessment. And the studies in regard to AI show that learners who use an AI-based tool showed greater improvement in vocabulary acquisition and reading comprehension than those using traditional teaching methods. I mean this is some pretty compelling research.

If you think about it, you read this, and of course, this is exactly what I need to be doing with my students to help them to grasp the language. AI was frequently used to assist students in learning, writing, reading, vocabulary, grammar, speaking, and listening. So that's just-- OK.

So now let's get to the lesson plan. Search engine queries for ESL education. So what I created was a lesson plan for them to compare a search engine running their Wh-questions through a search engine and then running it through chatbot GPT. So that they can compare for themselves the type of results that they get.

So before we even open up the computer, before we even go into the computer lab, I have the students generate as many Wh-questions and answers together in class and then they're using them together. Yes

Speaker 5: Just out of curiosity, you are using ChatGPT in your lesson, or did you have ChatGPT create this lesson?

Barbara Van Dyken: Yes and yes.

Speaker 5: OK, good.

Barbara Van Dyken: OK. Good. Because I'm not going to-- I'm not pulling the wool over anybody's eyes here. You can create lesson plans using ChatGPT and they're pretty doggone good, pretty cool. Yeah, pretty cool. And then you can adjust them and tweak them to fit and cater to the population of students that you're working with.

So before we even turn on the computer, they're generating Wh-questions, they're using them with each other to discern the difference between one time Wh-question where, which would tell them the location. English teachers, teachers you know this. So they've done this, and then we go into the lab and then the computers are guided through a Google Search with a list. Just prior to turning on a computer, they make a list of potential queries that they would like to conduct.

And so this next video is a demonstration or it is a demonstration of that. Let me get that.

[video playback]

- Hi, in this [ INAUDIBLE ] OK. So I haven't [ INAUDIBLE ]

This demonstration, I'm going to show you the difference between a search engine query and a ChatGPT query using Wh-questions to query the computer. And just for clarity, when we ask the computer a question, we are querying the computer. When we ask a person a question, it is a question.

But I'm going to be using these terms interchangeably because it can get very confusing, and I may just refer to the query in this demonstration as a question. Let's take a look at the search engine. In this case, we're using a Google Search engine. And in here, I would guide the students to this page. Hopefully, they will all be on their own computer and then we will either be doing this in Zoom or on a projection screen in the classroom, in the hybrid classroom.

So let's start with-- the very first question is, who. Who is? And we're going to say who is the president of the United States? That's the most common question I get with my students when I do this activities. Who is the president of the United States, and it returns Joe Biden. What the search engine does-- and this is how it differs from ChatGPT.

The search engine it relies on key words and phrases. And in this case, we're doing it with a Wh-question. It uses keywords and phrases and then returns the relevant info based on your keywords. And in this case, it brought back Joe Biden the president of the United States. And over here on the right, it gives a text just some lovely text that you can use with the students to further the Wh-question lesson.

So you can have them read and do some pronunciation, breaking up Joseph Robinette Biden Jr. And then highlight different parts of the text and have them generate even more Wh-questions through what you highlight. If you highlight a date, then they'll have to come up with a when question. When was he vice president? And so that sort of thing.

Now the next one-- just to-- I want to be brief in this so we don't take all day. So the next question would be a what question. And what is the name of the highest mountain in America? And it returns. It goes out. It's very quick. It goes out, and it returns Denali.

And it has these wonderful, beautiful pictures of Denali. And I will even take this opportunity to show them the different Google menu items up here, so they could look at images of Denali, of the mountain, and scroll through there to stimulate their imagination. And then use the Back button to go back to our original query.

Now the most natural question that comes after what is the name of the highest mountain in America and after the return of Denali-- well, quite naturally, we're going to want to know where is Denali? And this is all-- it's all guided. The students follow along, and we take the time-- So Denali is in Alaska. And what is it-- what's Denali also called? And this gives them the opportunity to read together and parse out the meaning of the text that's before them.

They might take a look at a map. Click on the map in the Google menu and get an exact location of Denali. And even run a few little searches or run the map in such a way that they could see where they are in relation to Denali. These are three questions who, what, and where using Wh-questions.

Now, you've got who, what, where, when, why, how, how much, how many and they can go through all of these. There's a whole worksheet that I've provided in my link for you to-- and a lesson plan for this. So let's compare it now to ChatGPT.

ChatGPT is different from a search engine query. It's a whole different animal in that it's an AI powered conversational agent, which means that through their handwriting the students can actually have a conversation with ChatGPT. However, in this case, we are just using Wh-questions, and we're being very targeted in our queries with ChatGPT. So it'll be returning shorter answers.

So the first question was, who is the president of the United States? And you hit Return, and it comes back. It's pretty much instantaneous. I think it might be a little slower than the search engine. I haven't done any timing with that yet. But that's probably the next thing I'll be doing.

So, OK. So it returns this answer as an AI language model. I do not have a real time access to current events or news articles beyond my cutoff date of September 2021. But as of my cutoff date, the president of the United States was Joe Biden.

And so this is providing some really good information. If you are looking for information that is more current, you're not going to find it in ChatGPT. And that's OK, for our purposes. Because we just want to generate a little bit of text to give the students the opportunity to practice their Wh-questions.

OK. So what was our second question that we asked in the search engine? I think it is what is the highest mountain in the United States? I think it was America. But we'll say in the United States. And you hit Enter, and the highest mountain in the United States is Denali formerly known as Mount McKinley. And which is located in Alaska. Its summit has an elevation of 20,310 feet above sea level.

So when I'm in the classroom or in class when we're doing this activity, I will highlight parts of it and formerly known as Mount McKinley. And I'll say, OK, students, what is the Wh-question that goes with this answer? So that's two so far. So we've got who, what, and then where. So our natural question, follow up question where is Denali?

And there it is. Denali also known as Mount McKinley. It's located in the Alaska Range in the state of Alaska, United States. So this is pretty much how I am using these Wh-queries. Our questions are becoming queries. I'm having the students generate as many Wh-questions as they can.

And then trying to show them the difference between questioning people and querying a database. Now, there's one more thing that you can do with this. And I would be remiss, if I didn't show you. And that is assuming that you don't already know. But with ChatGPT, you can ask ChatGPT to write a story. So you just say write a story about two mountain climbers climbing Mount Denali in the spring.

OK. But I just want to add one-- OK. Write a funny short story about two mountain climbers climbing Mount Denali in the spring. And then we click on the Search or the launch the rocket. And once upon a time, there were two adventurers mountain climbers named Jack and Jill. And if you want to know the story, you're going to have to-- you are going to have to link. I'm going to have this in the link of my presentation.

[end playback]

Barbara Van Dyken: OK. So if you want to know the rest of the story, you've got to click on the link. OK.

Speaker 1: Did they break the--

Barbara Van Dyken: It's a trick. That's a trick. So the different types of AI tools include virtual assistants like Siri, and Alexa, and Google. And we're already using these. If you want pizza and you say, hey you, Siri, pizza near me. What's a scary do? Nothing, sorry. [ INAUDIBLE ] We be talking to our devices-- we're already doing this. Yes.

Speaker 3: Do you use Alexa in your classroom?

Barbara Van Dyken: I don't. OK. Good. Thanks for that question because it was a no question-- I have to go any further.

Speaker 3: Yeah, there you go.

Barbara Van Dyken: OK. Wait, you've waste my precious hot air. OK. So chat bots, which can answer a query-- answer inquiries, provide technical support, or help with other tasks. So how many of you have called your favorite big corporation, that doesn't care and they don't have to, then you get a chatbot that's talking to you. Hi, welcome. Thank you for calling. And you can't discern whether it's a real person or a chatbot. So that's another type of AI tool.

Language learning apps which use AI to provide personalized learning experience and adapt to the student's individual needs and skill level. And that's something that is in the realm of Matt's expertise. And so he's going to present on that in a moment.

It's very clear that AI technology has the power to effectively transform the way that we teach adult language learners, and the way that we teach in general. Even adult learners or anyone for that matter, we're helping them-- with our language students, we're helping them not only to acquire a new language, but we're teaching them vital computer skills to survive in a new and rapidly changing environment and culture.

So to explore the topic of AI language learning tools further, I am excited to introduce our next speaker [ INAUDIBLE ] Matthew Sussman who I had the pleasure to meet last fall at the CATESOL Conference in Pasadena, and where he introduced his groundbreaking work with adult-- I mean, with AI powered speaking practice application called FLOW Speak, which is an innovative tool that's changing the way that learners practice their English pronunciation skills.

And it offers some flexibility to be able to practice anywhere at any time. Matthew will share with us FLOW Speak, and how it's contributing to the evolution of language learning. So without further ado, let's warmly welcome Matthew Sussman.

Matthew Sussman: All right. Good morning, everyone. It's such a pleasure to be here. Thank you very much for that warm welcome, Barbara. And I was really impressed with the opening plenary presentation speech yesterday by Dr. Vedder, and he started with gratitude. And I'd like to start with gratitude too.

I'm super grateful to Barbara who came up to me at the CATESOL Conference, really enthusiastic everybody knows, and just really leads with compassion and humor. And just so grateful to be connected with Barbara and have this opportunity to come and talk to you about what we're doing, share a little bit about my background, and then I have a special offer for everybody in the room, if you're interested.

OK. So FLOW Speak uses voice and AI technology to help newcomers, students, professionals to speak English more confidently. A couple of-- actually just last week, I spoke to 100 students online from Indonesia. They're about to come to the United States, and they are nervous. They got scholarships to come here. They're excited. They spent years and years studying English, so they've got all of the foundational skills.

They pass all the tests to get in here, but they're wondering if they're going to fit in. If they're going to understand. If people are going to understand them. And so before departure, I was trying to give them some encouragement and also explain the difference between textbook English and real everyday English. So I asked the students have you heard some of these expressions before, you're crushing it. It's all downhill from here. Give me a ballpark figure.

And you might-- I'm curious if anybody noticed that Barbara dropped quite a few when you were talking. I don't want to pull the wool over your eyes. We miss these expressions in our everyday life. And we just take it for granted. We just use it for humor. We use it to illustrate a point. These are super hard. I'm probably talking to the choir here, but these are super hard for non-native speakers to follow.

So I did this little exercise with the students from Indonesia. We're all on Zoom, so there's all these little small squares. And I asked the students, I said, OK, if you think you're cracking it is a good expression, put your thumb up. And if you think it's bad, put your thumb down. And out of 100 students 90 students went like this, right? And you can understand why, right?

And I say to the students, why do you think that? Crushing that's-- you're destroying something, right? So it just illustrates the point that English is really confusing, and we say things that sound like it's the opposite. And so even if you study a lot of textbook English and you come here, you get a shock. Because it's like it doesn't seem to make sense.

So Barbara had a great video, and I had to up my game. So do you guys know this gentleman, a comedian his name is Ismael? He's from Finland. Yeah? Yeah? OK, yeah, he's hilarious. So this is Ismael, he's talking about his favorite word. Kind of, illustrates the point.

Speaker 5: No, he's not.

Matthew Sussman: Yeah, yeah. So it illustrates the point-- Site can't be reached.

Speaker 3: It's blocked because--

Barbara Van Dyken: Because Instagram.

Matthew Sussman: Oh, Instagram is not allowed?

Speaker 5: Yeah.

Barbara Van Dyken: Yeah.

Matthew Sussman: OK, then I got to perform it. All right. Well, it was a great video. Check it--

Barbara Van Dyken: Maybe got on YouTube.

Matthew Sussman: Yeah, just check it. Look for Ismael on YouTube or something. But he does this-- he does a bunch of different humorous skits all around learning English, understanding like American culture. It's really funny. So I recommend you check it out. He does this-- he does this bit about his favorite word.

And he says Americans have this word for when they've had enough coffee, and then he goes like-- and he says-- and he said, if you start pouring the coffee and you go ta, ta, ta that's enough. Ta, ta, ta like that. Anyway, you got to check it out. It's funny much better than how I can do it.

But it's just another illustration of a newcomer coming to the United States trying to figure out our culture, trying to understand the vocabulary. We're trying to bridge the gap with our technology between this textbook English and the real everyday English that we're using all of the time.

And I think traditional publishers have shied away from this because when you're printing something, you don't want to have a mistake. There's a lot of nuance to it to explain it. And it can be dangerous-- it feels dangerous sometimes because if it's used in the wrong context, then maybe it's going to sound offensive or something. And it's also called slang sometimes. It's not proper, but is what we use all the time.

And we're-- I think we're doing a disservice to people trying to learn because it feels like two languages. There's the English that you learn in school and there's English that you hear when you have to do a Zoom meeting, when you have to do a job interview, when you have to try to build up relationships with co-workers.

So to tell you a little bit about my story, I've been working in international education for 25 years. I just came back from Japan. And I was working with the Fulbright scholarship program there. And I partnered up with John Goodman, who's a technologist and he's been working in the tech industry for 15 years. And we pitched our idea to some local investors here in San Diego. And our company is in Kearny Mesa.

So we're just a startup. We've been at it for a year and a half now. And these two gentlemen are ex-Googlers. They liked our idea and they gave us a little bit of backing. We have some professors from San Diego also guiding us.

We're using the technology-- and what I bring to this equation is that my experience with international students, sending people to work overseas. And so I think the best use of the technology is to build conversational fluency, being more confident, being able to follow the conversation, and be able to contribute to the conversation.

And we do that by using the voice in the AI technology and also adding in some everyday phrases that we use to help sound more natural. And the goal here is to help people achieve career success. So sometimes people ask me, well, what's your pedagogy? What's your learning philosophy behind all of this? And I can probably talk about in different studies and whatnot, but I just think really simply that learners need more practice.

When I was an English teacher in Japan, I could give plenty of reading homework. I could give lots of listening homework because there's YouTube and Instagram. And I can give lots of writing homework. But I couldn't give them speaking homework. And why is that? I thought about that as an English teacher why my students unfortunately were just not getting that much better at speaking.

They would come to my class once a week or twice a week, and English is fun. They learn a little-- they learn about the US a little bit. But I'm not really helping them in their career. And I thought about that. And the reason is the coming to my class twice a week is not enough, but I can't give them homework because if they go home they speak Japanese, or they speak Portuguese, or they speak Thai, and so they don't have the environment to use English on a daily basis.

So that's the reason why we make FLOW, so that people can have practice more often. The example I use frequently is if you take a piano lesson, the piano teacher-- any piano players here? Anybody? Maybe you took it as a kid? Or someone like me, your mom forced you to take it?

Piano lesson-- piano teacher will always tell me, she said, go home and you have to practice every day. Don't come back and take my lesson again unless you practice every day. You know that a piano teacher have that, kind of, Spartan image, right? Do it was like that-- it's like a basketball coach, right? You say, go home. Shoot lots of hoops. Come back again.

Every skill that we try to learn cooking, computer programming we tell everybody practice, practice, practice. But we don't do that with English speaking. Why don't we do that? Why can't we do that? Because there's some-- we know of some stories of success. Some students are super dedicated, and they'll try talking to the mirror or something. But ever they, oh-- that's an outlier situation.

So the idea of using this technology is just to have that unlimited speaking practice. And every time you speak, you get AI scoring feedback. And because you're practicing with a machine, there's no embarrassment. Now I don't know about you, but when I was an English teacher, I was careful about how much feedback I would give my students because I don't want to hurt their feelings, and I don't want them to not show up the next week. So I would hold back.

Yeah, I see some people nodding their heads. Like, oh, you said that wrong. You don't want to say-- and it gets culturally sensitive too because if you say, oh, you said-- it's like certain communities have a certain tendency to say pronunciation a certain way. And so it becomes sensitive, right? But with the machine, you get around that.

So these lessons on FLOW are short. They're, like, 2 to 7 minutes. We have vocabulary lessons. We have conversation lessons. We have ideas lessons. Idea lessons what's your opinion on this? Or what do you think? Or something that's more open ended. Conversation is a scenario between people talking. Vocabulary is learning like expressions like caught my interest or I'm a team player.

And then when you speak, you can get a breakdown of your score for pronunciation, speed, and vocabulary usage. Also it'll analyze-- so this is where the AI comes into play. It'll analyze your speech and give you a color coded correction.

So if it's green, it means it's clear. If it's yellow, it's moderate. And if it's red, it means it's not comprehensible. You can do a grammar check with it as well. And stay tuned because we're adding ChatGPT really soon, which I'll tell you how to say it better. So that's going to be cool.

So we've had-- we did 20 pilots last year. We have nine-- almost 10,000 people using it now from 29 different countries. So we're still in the testing, we're still learning mode, but we've worked with universities here and overseas, community colleges, companies, hotels. We're trying to reach all, kinds of, different audiences and see how people respond to it. And it's a continually developing process.

So what Barbara experienced lack at the end of last year, we've already made amazing strides since that time. But this is some of the feedback we get from our learners. I love the way he expressed this. He said, I used to feel stuck. So Dante is a researcher, very bright person.

Has been reading and writing in English a very high academic level but just didn't have the chance to speak English in Brazil. Came to the United States-- prior to coming to the United States, he's concerned about his English speaking ability. Said, I used to feel stuck like my tongue was locked. I love that expression.

And now that he can practice, he feels like it's a little bit easier to get the speaking process going. So we're not even really-- sometimes an expression like that it's like, it's not like we're really teaching English, we're just helping people unlock their potential with the English they already have.

Hitomi is that a Japanese language teacher at Spelman in Georgia. And she's emerged. She's here in the US, but the conversation is so fast. It's hard to follow native speakers. We drop all these expressions. And so she feels like with FLOW you can under-- practise these real conversations but at my own pace. I can break it down. I can slow it down. I can it over and over until I get used to it.

And one thing I'll add to this is that I envision this tool and all of the other tools as a supplement to the classroom, not a replacement. When I go into universities and I explain the platform, I think there's a little bit of hesitation amongst all of the teachers.

I think with a conference like this, there's probably a lot more people leaning towards adoption of technology. But I think when you go back to your institutions, you have the whole gamut, you have some people that love to try new technology, you get some people that really scares them.

But for me I think the technology is like a supplement. So in the classroom, you introduce the subject, you coach, and you're mentoring, you're explaining the nuance. But then you can use the technology to go home and practice, get in the repetition. Do it 20 times, 30 times, 40 times. Elaine is a virtual assistant from the Philippines says, practicing with FLOW has been less intimidating, more personalized, interactive, helping her in her career. So we did a trial with Grossmont. Thank you very much.

Barbara Van Dyken: Yes.

Matthew Sussman: And right after I met Barbara she's like, yeah, this is great. The epitomy of a early adopters. She's like, I'm going to try it right away with my students. So thank you very much. And so I gave Barbara like a two week promo code trial with their students. She distributed it with the school. She made a link on campus, so everybody could get to it easily. And this was our internal champion just to introduce it to everybody.

And we got 37 students who signed up. And in between October to November, everybody used it for a couple of weeks. We did a survey, and 100% of the students said it was convenient for speaking practice, and confidence improve, and they would recommend it to a friend. A short time period, but really happy with the results. So before I do this--

Barbara Van Dyken: Matt, there is a question.

Matthew Sussman: OK. Great.

Barbara Van Dyken: Online audience. I suppose we recognize and value different English act and accents. Is there a chance of a built in bias that value some ways you pronounce English more than others?

Matthew Sussman: Right, right. Good question. Thank you. Hopefully, you can hear me. So currently we're trying-- well, we're using American English just [laughs] very basic, very common expressions. But as we build it, we want to add in British English and Australian English and expressions from different parts of the country.

It's really interesting experience being a part of this. I gave a presentation to a gentleman from Nebraska and he shared that they have this expression called whipping donuts. And it's about when your car is like In the snow and you go around in circles or something like that. So we know areas have their own expressions and their own way of saying things and pronunciation, and so we want to build that out.

So if somebody in the US goes to the South, or they go to New York, or they go to Boston, or they come to California they can learn some of the lingo then that goes with that. So that's part of our plan. Yeah, part of our plan.

Speaker 4: I have a question.

Matthew Sussman: Yeah.

Speaker 4: Does those-- since I don't know the program, so does this have a certain topics they can choose topic and talk about it. Or they can just say, I want to practise talking about family, and then the [ INAUDIBLE ] start.

Matthew Sussman: Yeah, yeah. So let me do the demo. And I was trying to find the keyboard so I can--

Barbara Van Dyken: No, it's over here.

Matthew Sussman: There it is.

Barbara Van Dyken: I am trying to help with the Zoom.

Matthew Sussman: Oh, OK. OK, cool. So let me get out of that. And this thing in here. So I've logged in and we've learned so many things by trying to build the app and use the technology and make it easy for people. One thing I thought was really insightful is that we're dealing with a lot of young people in their 20s from around the world who want to use English to improve their careers.

And one thing one really big thing we've learned is that, a lot of young people don't have email. Or if they have it, they can't remember their email address much less their password. So a lot of people got-- it kept getting stuck. Out of maybe people you already have this experience in your life. A lot of people were getting stuck and they couldn't get back into the app. And so we send them a password reset thing, and then you've got that whole trying to catch up.

Anyway, long story short is we made a phone number login system. So off, if you can just remember your phone number, just put that and you can get in. That made a big difference. So let's say-- so you ask the question like how are the lessons grouped?

We group these into different themes. So this is a series called perfect, you're hired. It's about language for job interviews. But we have another one like on cultural foods. So we have a mix of casual situations and career development type of content.

And we are working towards making adaptive learning, so that the learner can get lessons that is suited to them. When they on board, they can assess what their level is and then they'll get less in that area. And going forward, they'll get lessons on the content. And as the AI is reading their responses, they'll get lessons that fit their level. Does that help answer your question?

Speaker 4: Yes. And then so it seems like currently there is no lesson for level like the beginning locally--

Matthew Sussman: Correct. So this is for maybe intermediate low. It is a starting point. So as you can see, everything's in English, so they have to have an ability to read and to listen and to be able to follow. So I recommend it for intermediate. Some of the lessons will say beginner, and there's some very basic stuff in here that people can play around with. And we have a beginner course that they can go through.

But I think overall, to get to this stage to be able to practice your speaking, you have to have the fundamentals for it. You need to have a good base first before you get to this. So this is something for starting from intermediate low, I think is what you would look at.

So for example, there's all these expressions we use when we're doing a job interview like it caught my interest, being a team player, two weeks notice, wear different hats. So let's do the lesson, stay on top of things.

Oh, yeah. Thank you. So I have this one. It's disabled right now. But I could put autoplay, and it would just automatically start loading the audio. But I'll click it here.

[audio playback]

- Stay on top of things. Stay on top of things means that you pay close attention to the matter.

[end playback]

Matthew Sussman: So in the classroom you could do a lesson on this. Or you could spend 5 or 10 minutes introducing the topic or the expression stay on top of things and how funny that is and visually show it to students of what that means. And then you could practice it in class. You could mimic scenarios for that. And then you could assign this for homework and say, OK, now go home and do this five times on your own. So--

[audio playback]

- First, practice speaking the phrase. Say, stay on top of things.

[end playback]

Matthew Sussman: All right. So I'll just tap here one time. Stay on top of things. I tap a second time and then within a few seconds my voice is heard, converted into text, and record it. Stay on top of things. And then I'll get a score.

And gradually, it gets more and more challenging. Now say it two more times. Stay on top of things. Stay on top of things. I used to do that often in the classroom just repeat the vocabulary say it over and over. So mimicking that. I got an excellent score this time. I actually recommend students keep doing it, until they get the excellent score.

When you do household chores, I stay on top of-- OK, so when I do household chores, I stay on top of things. So it goes through some examples like that, and then it'll ask you questions like you would see in a job interview toward the end of the lesson so that you can use it in context, right? It gets more and more challenging.

If I click on the score, it'll give me a breakdown for the pronunciation, the speed, and fluency. The technology is still developing, so you might be wondering why aren't you getting 100%. My English is bad maybe. I'm a little rough.

So I tell students aim for over 80. Excellent scores over 80%. The next level down is good, is about 60 to 80. And then below 60 is like an OK score. We don't give out any like that was terrible feedback. And if we go to the bottom here, you can see each word is analyzed for pronunciation. And then you get a breakdown for each sound. So household, so house hold. And I got excellent for each of those sounds.

And then I can do a grammar check, even though this is provided. It says we didn't find any errors, so good job. But if you were to say your own response, you can say whatever you feel like here. And if you said it incorrectly-- I could say, I household chores stay on top of things. I skip some words, Speak a little bit more stuttered.

See what-- OK, my score went down. Let's see if grammar will be effective. It says speed up. So my speed went from 99 to 65. It didn't find any errors there. I household chores-- well, I didn't catch it. And the next thing that we're working on is like the ChatGPT will analyze this, and we'll give you back even another way or another option for saying that. So that's how it works.

Speaker 4: So--

Matthew Sussman: Yes.

Speaker 4: Question

Matthew Sussman: Yes.

Speaker 4: More like a set of questions asked by the program and then the student responded?

Matthew Sussman: Correct.

Speaker 4: And then it's more like they're analyzing your speech and they give you feedback?

Matthew Sussman: Right.

Speaker 4: It's not really like really conversation that you do with AI?

Matthew Sussman: Right, right, right. Exactly. We're not doing like open conversation where you just-- the AI-- or you just ask the AI any question and it comes back. These are all guided lessons. And they're grouped into themes so that there's a specific learning outcome from that.

I've done English classes where you just have open talk, and it might be fun, but there's not really any kind of learning at the end of it. I think with these set lessons is, you can-- just for 2 to 7 minutes really hone in on the expression. There's a specific outcome learning that they can get from that. And so that's how we're doing it.

But the technology is developing. We will get to a point where you can have a conversation, a free conversation with the technology. But we-- currently, this is the format that we thought would be most successful for learners.

We'll also be building out some more features for pronunciation corner. So if there are certain words that they're struggling with, they can practice just on those. We'll also have a word bank, where people can get definitions and get more clarity on the words. There's other-- lots of other features here, I'd love to show you. And I think we're running short on time. Yes.

Barbara Van Dyken: Question from the online audience. Any thoughts about developing this for other language gaps?

Matthew Sussman: Yeah, thank you for the question. We are thinking about it. English will probably keep us busy for the next couple years definitely. But I think there's possibilities for other languages, French, Spanish, Japanese. So, yeah, I think there's potential here.

Barbara Van Dyken: One more. Is there an app for FLOW Speak. I think it is in QR code.

Matthew Sussman: Yes, so there is an app. So you can use it from any internet connected device. About 80% of our learners use it from their phone. Yeah, so there's these flyers and there's a QR code there. Or go to our website to access it at flowspeak.io. Yeah. OK, thank you.

OK. Any other question? Good for now? So one of the-- there's in the lesson, there's a button to say, I want to share my responses with the community. But I'll just show you what that looks like. So the AI is a developing technology. It gives you a sense of if you're on the right track or not, but it's not 100% perfect. So the way we supplement that is by having a channel here where you can connect with other people on the platform and members of our team to get feedback on your English.

It's a safe space. One of the things we learned, other than just getting people logged in with cell phone numbers, is that a lot of people don't have anybody to practice English with or share their excitement for English or feel it in their same level. And so we've created this space where people can leave their responses from their lessons and get feedback. So friends, and here's Joy. I'm ready to go with the FLOW. So she did a lesson.