[AUDIO LOGO] SPEAKER: OTAN. Outreach and Technical Assistance Network. JENNIFER GAGLIARDI: The next thing that I wanted to ask with him-- so we see the benefits that students are able to get information, but are they able to get advice? No. OK. So get information is good. Getting advice is no, OK? And then especially when it comes to personal identification. And now we were talking about this just a little bit before the session started is what equity issues arise as generative AI is used in teaching and learning? Yes, please. SPEAKER: Access. Access to it. And whether or not because there be the free form and then there's the pay for level. And those that can pay for the upper levels get more access and have more available to them, whereas those that can't afford to do that don't have the access just like with Wi-Fi and that type of stuff right now. JENNIFER GAGLIARDI: OK. So that is definitely a barrier. Yeah, OK. But on the other hand, we can use AI for doing representation. So to basically say, hey, my students do not look like this, let me generate a photo that looks a little bit more like my students, or let me generate a photo of a kitchen that looks a little bit more like my [INTERPOSING VOICES] kitchen. So anything else that we can talk about? OK, let's continue on. So that was a good discussion. Thank you so much. OK, so what is generative AI? Generative AI or artificial intelligence is an Intel technology that can create content, including text, images, audio or video when prompted by a user. So we've talked about using text. We've talked about using images. We have not talked about audio, and we have not talked about video. And so I think much more than the benefits or one of the advantages of TikTok over YouTube is that there are all sorts of templates, and that is a rudimentary form of AI-- how do we want to present this information. So I have not worked so much with the video or the audio. Has anybody use a video or audio? SPEAKER: No. Even using your Kindle to read to you. Anything, you're doing that. Yeah. Dude, if you have your Kindle book, you're saying, God, I'm so tired tonight. You hit that voice, and then the Kindle starts reading to you. And it's getting better and better. Initially, it was sounded really stilted, but we-- SPEAKER: I used a-- yeah, we went to a [MUFFLED VOICE] conference, and there was a presenter that talked about using a chat bot, making a chat bot that somebody could talk to and have a conversation with. And so I actually used it with one of my students, but it was just so frustrating for them because if they didn't say the answer right away, then it would go on to the next question. And it just-- I mean, this is not going to work for them. JENNIFER GAGLIARDI: Thank you for reminding me about chat bots because I tried so many times. When the chat bots just came out, I tried to use them or train it to prepare students for their naturalization interview. Tried and tried and tried and they would not-- they would ask the person's name. And do you promise to obey the law? And then they'll ask them the civic's questions. And it wouldn't be the current civics questions, which have been out since 2008. Oh, my God? Huh? But they would use the civics questions from, I think it was on 1998. So it was really kind of screwing up the students. So I couldn't even though I was really excited about ChatGPT bot because I think, especially like in Silicon valley, we have a lot of people, especially in the Indian community, especially with Indian seniors. And that was some of the information that was actually reflected on those USCIS LPR fact sheets that I was talking about there. They can read English. They were educated in English, but they don't speak English. So maybe, they've got a little bit of training about listening to American English from their grandchildren, but they need more practice in speaking. And so those are the tools that I really try to gravitate and been disappointed in. But now-- but that's in the past. What can we look forward to in the future? And so anybody can refer recommend some good chat bots. So we would really appreciate it. So let's continue on generative AI systems, create responses using algorithms that are trained often in an open source information such as text and images from the internet. So we'll get some examples of that. However, generative AI systems are not cognitive and lack human judgment. So if I ask something related to the naturalization, the naturalization interview or the natural naturalization form, they'll give me the topics, but they won't give me the questions. Although they'll give me a questions in such an obtuse way that it almost bears no relationship. So again, it's going to really take a lot of massaging. And then this is the key thing using prompts or questions to descriptions and to by a user to generate and refine the results. These systems can quickly write a speech at a particular tone, summarize a complex research, or assess access legal documents. Again, they can access the legal document, but they cannot give you legal information. And again, a lot of those legal documents that we're talking about, particularly things from USCIS, those have special security on it. And the chatGPT or apps cannot access PDFs that have special encoding in them. And now, a lot of our government-- yeah. Now, I've got a lot of our-- and they do it to protect personal information, OK? They're not saying, hey, we need to lock down these questions. They say, they're doing it to lock down people's personal information, OK? So it's not the premium on the questions. The government questions. Government questions are pretty much open, but they're locking down those PDFs because of the personal information. So again, we're going to have to be really, really careful on that. And if you see something that says this is direct questions from the N-400 application for naturalization, usually, it's really, really old versions of the N-400 application of naturalization, particularly for the N-400. They're using. They're going to be picking up things-- versions of the N-400 before 2014. In 2014 and again in 2016, that's when they added all those security questions. Those questions that are so difficult in part 12? That's the stuff that people really need practice on, OK? The 2014, the N-400 went from 10 pages to 20 pages and 21 pages and back down to 20 pages. So they're going to be looking at that. They're presenting that stuff. They're going to be looking at potentially, outdated or obsolete information. So we really, really have to take a look at that. Be very, very careful. But there's ways around that. And so we will get there. OK? So one of my favorite ways to talk about a frame, a prompt or a AI framework is TRACE. Task, Request, Action, Context and Example. So we have to define the specific task. And that was one of the things I was getting really stuck at. When I think about task, I think about a verb. And do I say, do I write? The guy yesterday, Jose or Joe, kept on saying the word "craft." And when I think "craft," hey, I'm thinking knitting. It's like, let's go to the-- where's the closest yarn store? That's what I'm thinking. But so do I want to use "make?" Do I want to use "write?" Do I want to use the word "list," OK? Request. We want to describe what we're doing. And so there has wanted to make a distinction about, am I asking for the questions that are on the form? Or I'm going to ask-- do I want to the questions that are going to be asked in the interview? There's two different versions. And for example, I'll give you an example. Were you ever a member of the military in the country? OK? So that's written in simple pasts. But a lot of people when they're asking these questions, they're not going to be asking it in simple paths like the form does. They're going to be asking it in-- they're going to be using present perfect. Have you ever been? OK? So just that really simple example is going to be something to be aware about. You're going to talk about the Action. So you're going to say, do I want to list this? Do I want to write this? And initially, when I was trying to do my prompts, I was saying, hey, give me all the questions on the N-400, it didn't work. So what I started to do is starting to focus in and focus in and focus in. For example, I would take a specific section of the N-400. I would say, hey, I want questions about from the job section, and I would give it the special section. That yielded some good results. But again, it was not really close to the form. I started saying, hey, I only want 3 or 5 questions. And 5 questions would give me a good sample of how, if this is a good prompt and if I'm getting good information. So that was things that I was looking for. And if you start limiting and start getting more specific, the more information that I could give them, the better it got the responses. The context-- I would have to say, hey, I'm looking at a form, or I'm looking at an interview. To test some of my information, I would say, hey, this is an interview going for job. This is a medical form. These are questions on medical forms. This is our medical history form. This is the questions that the beginning of your annual checkup. This is the questions that are asked during the annual checkup for a woman who's 65, OK? So give things more specific than trying to narrow it is basically the AI device was basically starting to pick up. Oh, I see where she's going with this information. So I would run through these things and then go to the to the AI-- the AI app. And basically, we're starting to doing comparisons and everything like that. And again, I think it's really important to give, not only the AI that context, but you wouldn't just simply hand a form to your students. You're going to build up. You're going to create so much context for your student that coming into this is simply-- this is going to be something new to learn, not something new to struggle with. So and then to give an example, and I would basically use, hey, look at a form, a job form for a landscape, or look at a job for the medical history form for a woman who's over 65, et cetera, et cetera. OK. So this is again, something that I learned from Jose. Some of our students have are in a situation, where they're coming to class, and they're tired. But you think, why aren't my students energized and happy to be in class? I'm happy to be in class. They're thinking about their kids. OK? You know, I remember when my dad used to go to school. It's like, why can't dad be home with us? OK? Well, he has to go to school. He's doing it for the family. So I wrote a history prompt. Yesterday, just really simple. And then I actually modified it. Write a children's story about a young girl, who resents her mom going to school to prepare for her natural naturalization. And then I added more and more and more to create more context for that. And I wanted to do this because our students are really mindful. They're adults. They have adult responsibilities. And coming in school, especially at night, really represents a sacrifice for their part. So this is the prompt that I put out. And then I wanted to basically show you three different comparisons of the opening line. So do you read the chat GPT one. This is the opening line. SPEAKER: Once upon a time in a cozy little house nestled in a bustling city, there lived a young girl named Lily and her mother, Maria. Lily loved her mother dearly, but lately, something had been bothering her. You see, Maria had decided to go to school at night to prepare for her naturalization interview. She wanted to become a citizen of the country they lived in. What comments do you have for that? What's your initial responses? JENNIFER GAGLIARDI: First thing is a story. SPEAKER: Yeah. JENNIFER GAGLIARDI: OK? It's a story. It's put together in a possible scenario situation. It has a bit of fictitious about it by saying, "Once upon a time." There's a conflict-- there a potential conflict between Maria and her mother about the Maria's decision to go to school to become the naturalized citizen. And then think about your own, what city you're from or-- SPEAKER: Fremont. JENNIFER GAGLIARDI: Fremont. OK. Are you going to do little houses on Fremont? SPEAKER: Yeah, there are. JENNIFER GAGLIARDI: No, I [INTERPOSING VOICES] students. Many cozy little houses. Are they living in? Are they living in apartments? Are they sharing houses with other families. So OK. So OK. So there's a little conflict. So I didn't catch cozy little house in a bustling city. Doesn't work. JENNIFER GAGLIARDI: Yeah, so you would sit there and you think about, would you see this or you think, hey, I can improve on that? OK. This does not look like Milpitas. OK? SPEAKER: No, but can I make it look like fremont? Milpitas? Yeah, I can take it from there. OK, let's see. Francisco, can you read and-- SPEAKER: Then one other like on the chat GPT. JENNIFER GAGLIARDI: Yeah. SPEAKER: Something along those lines. And I noticed that the names are always. Maria. They're like, it's so stereotypical. SPEAKER: Really, Lulu. If like, if I want to do that, like, for example, I'm getting a lot of students who come from Syria and things like they're not like-- it's not going to-- JENNIFER GAGLIARDI: So it's not going to mesh. SPEAKER: Yeah, exactly. So that's one thing yesterday that same activity, we noticed that when we would type something like I did like a low income or something low income and it gave me. Wy and there's, there's another bias in there because why would it bother Maria? To go to school for her naturalization process. Why would it bother her? Yeah, because, you see, Lily loved her mother dearly, but which gives you the indication that Maria's mother is not upset. Yeah, so-- let's yeah. SPEAKER: Yeah so for Claude, it says 8 boil her 1st control, Jenny sat at the kitchen table, swinging her legs as she finished her math homework. She looked over at her mom, Nina, who had her head bent over a thick stack of flashcards. JENNIFER GAGLIARDI: OK, we all know those flashcards. OK?But do what kind of response do you have to this? Totally. Different. SPEAKER: Very different than the first one. This is more like direct to the point. It's not giving you so much of the setting, I could say. It's just in the kitchen table. It seems that they're both studying. Mom seems frustrated. Jenny is maybe impatient or maybe wants help with teamwork. So, yeah, it's very different what chat. And again, both of these were created with a artificial intelligence. It just takes the information in a different way. Yeah, we think a lot of times now, people, instead of using their flashcards, a lot of times are you see our students scrolling on YouTube on those questions, OK. The same way the red flashcards. And then can you read the Gemini prompt? No Sure Here. Stared at her Bowl of cereal, the milk swirling and appetizing. Her mom, hunched over a thick book, mumbled to herself, scribbling furiously. Every night, this happened. Stacks of papers replaced Lyla's bedtime stories, and furrowed brows replaced warm goodnight kisses. SPEAKER: There's a lot of description going on. It's very dramatic. It's unappetizing. Curiously, it's furrowed brow, so. JENNIFER GAGLIARDI: How many real meal prep goes into cereal, you know? that's like last stitch. But it's interesting that they're set- that the scene is set for studying, and different stacks of paper replaced bedtime stories. It's a really interesting image. OK, I'm just-- SPEAKER: What was the prompt again? JENNIFER GAGLIARDI: The prompt was, write a story about a young girl who resents her mother because the mom is going to school. And WAS doing this-- I was trying to write this story. Oh, let me give it to you. After many months, the girl goes with her mom to her naturalization interview. She sees how her mother overcomes her nervousness. Pass the naturalization interview, and is proud to stand with her mother at her Oath of Naturalization Ceremony, OIC. A couple of grammar errors. Anyway, I want to show the closing. And so in most of the stories, the girl goes with her mom to the interview. The mother-- she sees how her mother basically takes a couple of deep breaths or twisting a handkerchief or something like that but is able to get through. And now this is the last part. Can you read the ChatGPT on the top part of that? SPEAKER: "From that day on, Lily knew that no matter where life took them, she and her mother would always be together, facing every challenge with courage and love. As they walked out of the ceremony hand in hand, she knew that their bond was stronger than ever before." JENNIFER GAGLIARDI: Yeah. So, good ending to that, especially that they know that there's a continuation. This is not the last part. OK, how about the top one? SPEAKER: "As Nina waved her little American flag, Jenny went over to give her new citizen mom another big hug. She was so glad all that hard work and studying had paid off. Now they would have more time for cozy nights at home" together. JENNIFER GAGLIARDI: Bringing it back together with being with the family. And my mother was with her mother-- OK. So my mother was born in 1935. Her mother naturalized probably in 1942 during World War I, and they were German-Americans. And so in Illinois-- and my mother was beaten up regularly just for speaking German. But she really remembers going to that interview and participating in it. And she was so proud-- she was of her mother. Can you read the Gemini part SPEAKER: "The resentment melted away, replaced by a deep sleep pride. Her mom, with her dedication and resilience, had not only become a citizen, but also shown Lila the power of perseverance and the true meaning of home. The chipped blue door, once a symbol of frustration, now stood as a symbol of their shared journey, a reminder that sometimes the most beautiful changes come with a bit of struggle and a lot of love." JENNIFER GAGLIARDI: [INTERPOSING VOICES] SPEAKER: I love her [LAUGHTER]. [LAUGHTER] JENNIFER GAGLIARDI: So I do have a connection in there that you can click them [MUFFLED VOICE]. You can read the entire story as you have this in the notes. But it's basically one of the things that the store was-- that appears earlier in the story. They have to pass through that door all the time. And so it talks about going on to anything, and how she used to be really ashamed of that door. And now it's a real symbol of passing on to a new day. But I thought that was really interesting how they use symbols in that. So I know I have gone probably way too much on this, but I think it's really important to basically create some materials. So I'm going to be working a little bit more because it's kind of like we have so many stories, but I need to make something better for my students because sometimes they lose heart and forget why they're in class. And especially, when so much is pulling away from them. And that they know that it's only going to be a short time, that they'll be able to continue on. OK. I want to share a resource from OTAN and otan.us. And we have our resources. And this is a really great article about streamlined planning with AI crafting custom lessons. Was anybody able to participate in some of [MUFFLED VOICE] cultures workshops this weekend? Anybody go to-- she constantly is doing web training and everything like that. And she's very good at creating lesson plans and has used-- created lesson plans. Ultimately, she always incorporates some aspect of technology in there. So please go ahead and take a look at this one. The permanent link is, it's Details 201. So this is the 201st story about this. But this is a much, much, much longer article filled with a lot of connections. And she's-- oh, excuse me. She connects it to all these different wen-based class activities. So she has web-based class activities created with AI for adult basic education, career, and technical education, ESL, and citizenship, high school, GED, and holidays, older adults, and all activities. Let's see. Francisco, just walk back in the door. Francisco, did you have anything to say about Susan Coulter's-- this article by Susan Coulter? FRANCISCO PINEDO: It was-- JENNIFER GAGLIARDI: She just posted it to OTAN. Or were you in one of her sessions yesterday? FRANCISCO PINEDO: I wasn't, no. JENNIFER GAGLIARDI: OK. OK. So, anyway, please take a look at some of these lesson plans. That's the key thing that I wanted to share from this. So, I'm getting back to-- I want to move on to some of the prompts that I have written. And particularly the prompts because I'm in an ESL or EL civics class. I'm very, very oriented towards the life skills. So some of the students said, hey, this class, English class is so different than the English class we had in our own home countries, which was grammar, grammar, grammar, or vocab. And here we are. We're talking about changing tires or filling out forms or whatever the case may be. So here's the prompts that I wrote-- questions, comment. I wanted them to list questions, comment to school registration forms, employment application, medical history forms, and USCIS application forms. And then, going from what-- so I'm making a decision here between forms and then questions. Make a decision between that and then questions, comment. Just two interviews. So school registration interviews, employment interviews, in-office medical exams, USCIS interviews. And it was really interesting to see how the topics, different topics came up, and the different grammar points that came up. So I'm going to basically step out of this slide. And hopefully, I can go to my ChatGPT account. Share screen. I'm going to go to, I think, this one here. And I'm going here to here. Naturalization in here. So what I had done was I had used different prompts to talking about-- let's see. Where's the different prompt? Prepare for a naturalization interview. 100 questions. I was not getting-- I kept on saying, hey, where are these questions? I'm getting explanations about things that people are being explained-- that they're explaining. So, eventually, what I started doing is saying, hey, I want to limit the questions that I'm asking 5USCIS N-400 Part 12 Questions CEFR1. So I'm saying, hey, give me something in basic English what they're going to ask. OK? I wanted to add that because I wanted to get something low level. But I would say that somebody from ESL maybe-- maybe the second semester ESL 2 or ESL 3 would be OK with that. What are you seeing with that? Do you think that's for ESL 1. No. The grammar is way too much. So how did you compen-- how do you compensate for levels when you're asking questions? FRANCISCO PINEDO: So what we do is-- well, after they take their CASAS 4, their CASAS test, we look at their level, their score. And then on the CASAS site, there's a document that correlates their CASAS reading score to grade level. So then when I use ChatGPT, I put in, for example, show me examples of this at grade level five, for example. And then it will populate the question, thinking it's for a fifth grade student, when, in reality, it's for my ESL student. Because if I put for ESL, it's still going to be high. So I always use that grade level conversion to make-- for example, on mine, I'd say, create a worksheet with 15 fill-in-the-blanks on past tense verbs for grade four, which would be my low intermediate class. So then it would populate it, and it would create it with vocabulary that when my students see it, they're like, oh, yeah, this is good. There might be one or two words that might be a little bit too high for them, but it's better than if I just put, create for an ESL student because then it's going to be, like, OK, give me the whole thing. JENNIFER GAGLIARDI: I really appreciate that. The other thing is, is I was looking for-- I was trying to put in the NRS levels. And yeah, that wasn't working. National-- yeah. And that's really important when plus had just updated those levels. So that's going to be really important. And I'm sure as ChatGPT gets more advanced, we're going to be able to find that-- we'll be able to find this information. What I'm trying to do is go in as I'm scrolling down. And maybe I have scrolled too far. Oh, seven days ago. When I first started doing this and saying, give me information, what's on the naturalization interview, they were giving things way off. Sorry, way far afield from what's actually on the form. And stuff that wasn't even-- was not even on the earlier forms. So there was a lot of testing or training that was going through. And eventually I think it's going to take a little bit too long to go back to it. Eventually, what I had to do is basically list five questions that are common to job interview and naturalization form CEFR 1. So I started comparing the different forms. And I was only asking five questions at a time. And I really did-- my results starting giving me a lot of it from there. OK, I'm going to step back into my form. I think I'm going to have to-- it's obvious that I'm going to need to start organizing my prompts a little bit better. OK, I'm stepping back into my presentation right now. I wanted to go from there. So you're basically using it to create content for your classroom. I'm doing it because I want my students to go from their form to interview practice. And especially, I want to do different versions of the interviews to share for the students. Now, I'm going over to the forms and interviews. Is that right? Yeah, forms and interviews. OK. Write a lesson plan for a B2 class in which the students complete employment application, then role play a job interview based on employment application. And it was a really bare bones one-hour lesson plan. But the thing is, is that it was also-- it had a warm up. It had everything that I wanted. But it was just OK. And I feel like I could take that because a lot of times I'm so tired and thinking, how do I do lesson plans again? And then it's, like, oh, OK, if I can use this as a beginning and then go back and maybe help it get back into [MUFFLED VOICE] you say it would be a form? SPEAKER: Yeah. JENNIFER GAGLIARDI: And then move on from there. I can add all the resources and put it in all the links that I want to. It's going to save me a lot of time. SPEAKER: Is B2 a universal term? JENNIFER GAGLIARDI: B2 is based on the CEFR. So again, this is intermediate. That's intermediate. SPEAKER: I don't teach it. JENNIFER GAGLIARDI: That would be probably-- yes, [MUFFLED VOICE], OK? SPEAKER: OK. JENNIFER GAGLIARDI: So, the next thing that I wrote was, write four sets of 10 USCIS N-400 applications for naturalization employment and education questions. So if you're on the N-400, there's a certain section about employment and education that are very similar to what you're going to ask on a job application. And they're also going to ask you questions about have you had any arrests? Do they ask you questions about job skills? No. But it's a way for students to move from what they've learned previously in an ESL class about jobs. And now they're moving on to this other thing. So they're able to take this information with them. And then finally, I asked, hey, write this lesson plan, incorporate these four sets of lessons, and then take the role of USCIS officer to interview another student. So you're basically adding more and more amongst them. OK. Finally-- and I do have examples of that-- You can dig into this. And I have them all in the notes for the slide. So if you go into those, I have all the material there. I think one of the most important things about AI is that we can use it to basically promote equity and civic education. So, for instance, this is a really fascinating article that I cannot recommend for the uh-- uh-- It's from restoftheworld.org. And it's 2023 AI. It's how AI reduces the world to stereotypes. And so this was really interesting to see these different women with the American flag. I thought that was really interesting. Especially, a couple of years ago, you would see the image of women-- I'm thinking about the woman wearing the hijab of the American flag. And some people said, hey, you don't wear the flag that way. That's not what the hijab is for, et cetera, et cetera. So we're saying, hey, I think this is more inclusive, and they're saying, hey, that's actually offensive. So basically, we try. We're saying, hey, we're trying to be more inclusive. If it's not, we need to have a discussion about that. So, really I kind of recommend this article. And again, the link is in the slide. So what I've tried to do was try to use MS Designer to basically imagine moving from students. And you can see when you talk about students, everybody's young and everybody's healthy. Everybody looks really happy. I don't remember college that way at all. I don't know, do you guys remember college the way? I don't remember. All right, so we have a citizenship prep students. And so, again, they do not look like my students at all. But again, there seems a little bit-- they seem to be a little bit older. We're not clear who the teacher is. Is the woman on the front, or is it the gentleman in the back? The only way we know that this is a citizenship prep class is you see the big flag, but you can have a big flag anywhere. So you're going to have to add more information. I would definitely really kind of work on this slide. And then we have an image of new American citizens. And you take a look at this document down her. It basically says nothing. It's just chicken scratch type of stuff. So you would have to go in and clean that up. American certificates-- our citizenship certificates look nothing like that. Are those American flags or are they napkins that look like American flags? So you have a lot of things that you can basically work with. But again, this was just really quick prompts that I basically put in. And then, finally, this was something we talked about in the Statue of Liberty, which I went to see for my very, very first time last June. It was so cool. Oh, my god. I thought it was going to be a religious experience. It was an experience of being cold and damp and wet. But I remember one time being contacted in 2007 about a documentary. And they wanted to come and film my class, a citizenship class. I go, my students don't look like your audience. My students are all Asian. They go, that's exactly what we want because we don't have Asian people in the classroom. This is what we want, OK? So that's always stuck with me. So I tried to reimagine. Just put an Asian statue of liberty. I got this with lotus flower. We've got a lotus flower out here. I really love how we had the embroidery on the Statue of Liberty, The garment. And it's the right color. And so it's nice. Does she look Asian? Maybe a little bit. OK, yeah, we'll go with that. Let's drill down on what Asian. What kind of Asian? So in Milpitas, we have predominantly Vietnamese students, so I wanted to imagine them as a Vietnamese woman. That looks very good and what the eye, so that looks great. This one, one of my friends took a look at it. And they're like, what is that? We don't know. It's a lotus, and it's a dragon coming out of lotus. So they go, hey, we don't know what's going on with that. And then we have a Palestinian hijab. Again, they all have the right blue green color, but there's little details. Like, what's going on here with the-- is that the flag of Turkey? What's going on with that, the background? But basically, trying to get our students to see themselves in civics content is the most important thing that we can do, and that we can do as teachers. So basically, trying to make connections back and forth. And let's see, that's it. So if we have-- do we have any further questions or any further comments?