Speaker: OTAN-- Outreach and Technical Assistance Network.

Carolyn Zachry: Welcome to the Adult Education Office's Federal Program Monitoring prerecorded video. The California Department of Education is the state agency for WIOA Title II funding, and is therefore required to monitor our grantees for compliance to federal regulations, grant requirements, and assurances. While we are monitoring for compliance, we also want to shed a light on all of your successes. Get ready to take notes and make a list of all of your questions that can be answered at the live training.

Before we continue, though, I would like to thank Abby Medina Lewis and David Stang from the Adult Education Office, who are our federal program monitoring leads, and to OTAN for all of their tech support.

David Stang: Hello. My name is David Stang. My colleague Abby Medina and I would like to welcome you and thank you for attending the Adult Education Instrument Virtual Training for 2022/23. For those of you who don't know us, Abby and I are Education Program Consultants in the Adult Education Office, or AEO, at the California Department of Education.

This training will focus on Federal Program Monitoring, or FPMs, and more specifically, the Adult Education Instrument for the upcoming year. Speaking of the Instrument, you should all have a copy of this year's Instrument with you today. We will refer to it often, and you will want to follow along and take notes. If you do not have a copy already, please pause the presentation, and download or print a copy. You can find the Instrument at the first link of slide 32 under Resources. Abby.

Abby Medina Lewis: Thank you, David. The purpose of today's training is to define compliance monitoring, what it is and what it's not. This session will also provide you with an overview and understanding of the CDE's Adult Education Program Instrument, which is a tool used to ensure agencies receiving WIOA Title II AEFLA funds are in compliance with federal and, where applicable, state law. Lastly, this session will provide clear expectations on the specific evidence that are required for each item on the Adult Education Program Instrument.

So today's agenda includes federal program monitoring, as stipulated by the Workforce Innovation and Opportunity Act, Adult Education, and Family Literacy Act, which is also known to you as WIOA Title II AEFLA program. We will look at the Adult Education Program Instrument items, and review evidence requests that you will be expected to provide and upload into the CDE monitoring tool, also known as CMT. Finally, we will provide you with some useful resources to help you with your review.

Now, just as a quick side note here on CDE monitoring tool or CMT-- the CMT training is a separate training, and will not be addressed in this session. The FPM office will provide the training on CMT. And details regarding CMT training will be shared with you all via email soon. In the meantime, we provided you with a link to the CMT web page in the Resources page of this PowerPoint slide, which we will show you at the end of this session.

David and I will also hold an interactive live two-hour question and answer webinar on August 25 from 10:00 AM to noon. Agencies scheduled for review this year must view this prerecording in its entirety prior to participating in the live Q&A session. Only those participants that have viewed this prerecorded session will be allowed to participate in the live Q&A webinar.

You will be required to register to participate in the Q&A session. We will email you the link to register for the Q&A webinar at least two weeks before the session. And finally, in the live Q&A webinar, you will also have an opportunity to hear from two agencies that were reviewed last year.

Guest presenters will share their FPM experience, as well as share strategies for a successful review. You will also have an opportunity to ask them questions. So please don't miss out in this Q&A session. It'll be very interactive, and you will also get to see me and David live on video.

And finally, I want to emphasize that viewing this recording and attending the live Q&A webinar on August 25 is required if you are scheduled for an FPM this year. We have found that agencies that did not participate in training and the Q&A webinar had a difficult time with their review.

David Stang: The WIOA Title II AEFLA grant requires CDE to conduct on-site and online monitoring of all adult education providers. Any and all adult education federal grant recipients will be selected for review at some point. On-site reviews are typically more comprehensive in nature. They are conducted at the adult education agency, and include a review of documents or evidence requests, observations of the program offerings in classrooms, and interviews with key personnel.

Online reviews are similar, but do not include much of what occurs in the review process at an actual site or location. The review of documents is the same in both processes. In the past, and because of COVID-19, most reviews during the last few years were conducted via online and/or telemonitoring. This year, agencies are scheduled to be reviewed on-site and online.

If anything changes, agencies will, of course, be notified. With COVID, of course, the possibility of on-site reviews being converted to an online or monitoring format is possible. The purpose of monitoring reviews is to ensure agencies are in compliance with federal and, where applicable, state laws. Current monitoring cycle is for this program year 2022/23.

Abby Medina Lewis: So what is compliance monitoring? Compliance monitoring is simply an overall determination of whether agencies receiving federal funds are meeting statutory program and fiscal requirements for certain programs. It is CDE's fulfillment of regulations to ensure that state and federally funded agencies meet minimum compliance. And you will hear throughout the course of this session us use the term "minimum compliance."

It is also an opportunity to examine and improve your agency policies and procedures. This could include, but not limited to, your assessment policy, no fees policy, time accounting, and other programmatic and fiscal policies. Compliance monitoring is not an audit.

We are not auditors, we are program reviewers. So we're not trying to get you. It is not a gotcha opportunity for us to take funding back from your program.

David Stang: All programs, whether they be adult ed, before and after school, or fiscal monitoring, develop program instruments to guide monitoring reviews. And within those instruments are categories or items that the reviewer looks at during each review. The Adult Education Instrument has 10 items or categories, which Abby and I are going through review with you today. Abby.

Abby Medina Lewis: Thank you, David. The 2022/23 Instrument is organized into 10 different categories or items. Each item has a respective list of evidence requirements that must be provided by agencies to meet federal requirements. So let's take a look at these 10 Instrument items.

The 10 Adult Ed Instrument items are AE 01, collaboration, alignment, and support services, AE 02, financial accountability, AE 03, data collection and program effectiveness. AE 04, staff qualifications and professional development, AE 05, needs assessment, AE 06, serving individuals with disabilities, AE 07, intensity, duration, and flexible scheduling, AE 08, evidence-based instructional practices and reading instruction, AE 09, effective use of technology and distance learning, and finally, AE 10, Integrated Education and Training or IET. These 10 instrument items may look and sound familiar to you, because if you recall, when you applied for the grant, these are the same as the federally imposed considerations that you were required to respond to in the WIOA Request For Application, or RFA.

So what are evidence requests? Evidence requests are examples of documents that agencies upload into the CDE monitoring tool, or CMT, to demonstrate minimum compliance. All evidence requests must be uploaded into CMT 30 days prior to the first day of a review. So let me highlight an example.

So for example, if the first day of your review is scheduled on November 1, then you are required to upload and certify all evidence requests by October 1. It's important to get all of your evidence into CMT 30 calendar days prior to the first day of your review, to allow the CDE program reviewer just enough time to review the evidence and provide you with comment on each Instrument item. The CDE program reviewer is required to provide updates for each item no later than 15 calendar days prior to the first day of the review.

If you are unsure about a particular evidence request, please contact your CDE Adult Education FPM reviewer and ask for clarification. Now typically, your region CDE Adult Education consultant is also the assigned reviewer for your FPM. However, there are some instances where a different reviewer is assigned to your agency, such as scheduling conflicts.

Now we will begin our deep dive of the 10 Adult Ed Instrument items that I shared with you in the previous slides. David, can you please start us off with the first item.

David Stang: Absolutely. If you would look at page 1 of the instrument, I'd like to bring your attention to the initial description or legal guidance section that you will see for each item. Item one, or AE 01-- collaboration, alignment, and support services-- includes all of the federal guidelines the Adult Education Office uses to guide the review for this item. We highly encourage you to read each guidance section for all of the items to provide you with an overview of the item-- that is, what the item is really about, collaboration, alignment, support services, for example. You should expect to see guidance in these areas.

The first guideline 1.0, for example, reads, "Eligible provider demonstrates alignment between proposed activities and services and the strategy and goals of the local plan, as well as the activities and services of the one-stop partners." Guideline 1.1 reads, "Eligible provider's activities coordinate with other available education, training, and social service resources in the community."

Secondly, the AE 0 included each guideline or federal regulation in the initial section of each item as a foundation for each evidence request that is required within the Adult Education Instrument. This is what we are looking for in this section. And these are the areas we are required to monitor for this item.

It makes sense that under AE 01, for example, we will see collaboration with other partners and providers, that we will see alignment with the local plan, and support services for students. In addition, if there are any findings during the course of the review, these guidelines are what we use to form the legal basis of each finding.

With slide 13, we're going to take a closer look at actual evidence requests. Each item includes a number of documents or evidence requests agencies are required to upload into CMT, the CDE monitoring tool. In many instances, we are asking for a very specific piece of evidence. Purpose of requiring evidence requests is to ensure recipients of federal funding meet minimum compliance. To do so, we ask that you upload the requested documents, or in some cases, the equivalent evidence, to meet compliance.

If you look on page 2 of the Adult Ed Instrument, the first evidence request you see is Memorandum of Understanding, or MOU. The description for MOU reads, MOUs are formal informal agreements that demonstrate current collaborations or partnerships with the Adult Education program. Agencies are required to upload at least one MOU, or informal agreement, with another provider contractor or agency that reflects the types of services being provided by each partner.

That is, how are the agencies collaborating? What does the partnership on behalf of students involve? Can agencies upload more than one MOU? Of course. I'd say two or three are ideal. In order to meet minimum requirements for this item, however, at least one MOU must be uploaded.

The description for the next evidence request resource guide indicates evidence that demonstrates resources available to adult education students-- resources such as child care, transportation, counseling, tutoring, et cetera-- for the current and prior year. Agencies should have available to students a resource guide of some kind-- a pamphlet, in some cases a catalog, even a web page can be used to meet minimum compliance for this request.

The point is, how do students know what services are being offered? What examples of support are available to them in the event they need child care, for example, a counselor or tutoring? If you also notice, this evidence request indicates agencies must upload a resource guide for the current and prior year. Some say current and prior three.

In some cases, if your review is scheduled early in the year, you may not have anything to upload for the current year. That's OK. Please be sure to clarify with your reviewer what, if anything, needs to be uploaded if you see "current year," and your review is scheduled within the first few months of the year. In this case, agencies should be able to upload a resource guide from the current year, since ideally, you would have that information available for students at the start of the year. But if for some reason it is still in development, your resource guide from the previous year would be sufficient.

The next evidence request under secondary evidence request reads, "Supplemental evidence of collaboration." This is not a required request. In the event an agency is having trouble meeting minimum compliance-- for example, an agency doesn't have an MOE or a resource guide available-- the reviewer in some cases might ask for supplemental evidence of collaboration to help an agency meet minimum compliance.

Having an MOU, for example, is a requirement. But in some cases, if an agency has both phase 1 and phase 2 of their umbrella MOU with an AJCC in place, a reviewer might be OK with an agency uploading supplemental evidence of collaboration if they don't have a current MOU in place. If you look at the description, examples of supplemental evidence could be recent meeting agendas, sign-in sheets, meeting minutes, et cetera. But only upload these documents if asked by your reviewer.

This brings us to the final evidence request for AE 01. The description for this evidence request reads, "A current signed MOU with the Local Workforce Development Board that outlines the activities and services of each partner and the overarching AJCC delivery system to support the goals of the local plan." This request includes both phase 1 and phase 2 of the MOU.

Some agreements may include both in one document. Typically, each phase is developed separately. But in either case, phase 1 should reflect all of the services being provided by each of the partners involved in the MOU, as well as the name of each partner, and actual signatures from the appropriate representatives at each agency.

It is important that you locate these documents now, and ensure that you have a complete copy of the MOU which includes signatures from all partners. The process to request a copy from the LWDB during the course of a review can be time consuming, and can even lead to a finding, if you are not able to upload this request before the end of the review. A little work on the front end goes a long way so that you're able to focus your attention on more important matters during the course of your review. Phase 2 should be an MOU with all of the cost-sharing agreements between all of the agencies.

Again, as we move to AE 02, please remember to read the initial guidance section for this item, especially since several additions to this section were included in this year's instrument. The initial evidence request for AE 02 reads, "Historical inventory list of all equipment purchased for $500 or more per EDGAR and ED Code requirements and a record of last physical check of items. If no purchases were made, indicate as much by posting a comment in CMT."

For this evidence request, you will also see, under item instructions, specific guidance related to equipment inventory outlined by the office, provide percentages of Education and Family Literacy Act funds used, serial numbers, resource codes, location of equipment, and items that were disposed of, and how they were disposed. If your agency has not purchased any equipment this year, be sure to indicate that in a comment. And if you have any questions about the format or how to organize the document, again, be sure to ask your reviewer.

Be aware and anticipate potential issues. Your reviewer will cross-reference any information your agency provided to the AEO regarding the purchase of equipment via ECRs, CIPs, or otherwise, with what you upload into the California Monitoring Tool. Let the reviewer know ahead of time if there are discrepancies, so both the reviewer and the agency have time to troubleshoot, if necessary.

The description for general ledger is a general description used by all program offices at CDE, detailed general ledger for the specific resource codes being reviewed. The specific instructions pertaining to the Adult Education Office are to include specific resource codes for Adult Ed. Those are 3905, 3913, 3926, and of course, 6391. These are all examples of the resource codes we would expect to see in an agency general ledger.

Ledgers from the current and prior three years should be uploaded into CMT. What reviewers are looking for primarily with this evidence request are to confirm the expenditures which were reported to the CDE, and the final ECR for each object and resource code, and two, to ensure that federal expenditures are reasonable, necessary, and allowable under federal regulations. The description for in-kind and other non-federal expenditures requires agencies to explain or document how expenditures are calculated and included in the final expenditure claim reports, or ECRs.

Again, what you upload into CMT should be reflected in each year's ECR. Your description can be a simple Word document which reflects your agency's actual in-kind expenditures in aggregate, and/or an explanation of what your in-kind expenditures are. Be sure to ask your reviewer if you're not quite sure what "in-kind" means, or if you're not the one who submits the final ECR each year. At the end of your quarterly ECR, there is a definition of what other non-federal and/or in-kind contributions are under the page 4 non-federal sources of funds expended for WIOA.

Agencies are not allowed to charge student fees for any program where federal funds are provided to agencies. Registration fees, fees for books, tuition, are all examples of fees agencies are not allowed to charge. Agencies should include in their catalog, or website, resource guide-- somewhere clearly visible to students they are not being charged fees.

Ideally, we encourage and would like to see an actual policy indicating students will not be charged fees in any federally sponsored adult ed program. However, as long as an agency clearly communicates it is not charging fees, it should meet minimum compliance. Agency's fee policies for the current and prior year should be uploaded.

Org charts should include the names and titles of staff and board members, reflecting leadership at the district and adult education level. Sample board minutes only apply to community-based organizations, including charter schools. Two samples from each year for the current and prior year will meet minimum compliance.

The description for Staff Funding Report indicates that funding percentages for all adult education personnel should be included. There is a template or example on the Compliance Monitoring web page at CDE, or simply ask your reviewer for an electronic copy. The report should be for the current and prior three years. Again, it is for all Adult Ed staff, not just AEFLA-funded positions. Reviewers look at this request over a three-year period to establish a trend-- that is, to ensure state dollars are the primary source of funding for the overall compensation of work performed by Adult Ed personnel.

Time and effort policies and procedures are written policies for documenting time and effort of employees that work on federal programs. Your agency's policy should include specific practices for documenting actual hours worked, as well as internal controls and employee training around accounting for time. The two most important components of this request are first, to make sure your agency has a policy-- if you don't have one now, get started on it right away-- and second, make sure that what is reflected in your policy is what is actually taking place with respect to time accounting. Please work with your reviewer if you have more specific questions.

Our next evidence request, Time and Effort Records, can include personnel activity reports, semi-annual certification, or other equivalent records. Documentation to support salaries and benefits charged for each program funding source under review should be provided. Again, please work with your reviewer to determine the number of samples for each quarter to be uploaded into CMT.

Supplemental Evidence of Financial Accountability-- this request was added to the instrument last year to provide both agencies and reviewers additional opportunities to meet compliance. Again, as Abby mentioned, the purpose of the review is not to afford CDE an opportunity to take money back from you. It is the agency's responsibility to provide the appropriate evidence or documentation.

It is your reviewer's responsibility, us, to assist your agency with knowing what evidence meets minimum compliance. In the event reviewers need more information to meet compliance, they may ask you to upload additional documentation in this area. Examples of supplemental evidence are payroll records, position control reports, and detailed ledgers for Object Codes 4000 and 5000.

That does it for financial accountability. Abby, are you ready to talk data collection and program effectiveness?

Abby Medina Lewis: Of course, David. So now we're going to pivot from fiscal to data. I will talk about AE 03 data collection and program effectiveness. The CDE requires all WIOA Title II AEFLA grantees to use the TOPSpro Enterprise, or TE, database system to collect and report adult learner demographics and program information.

In addition, the WIOA Title II AEFLA-funded agencies are required to comply with a National Reporting System, NRS, for adult education requirements. TOPSpro Enterprise, or TE, uses the Federal Tables to report statewide aggregation totals to the federal government. So for this item, AE 03, data collection and program effectiveness, we are verifying if your agency is meeting data accountability requirements in accordance with WIOA Title II AEFLA grant.

There are nine evidence requests related to data collection and program effectiveness, of which five of these evidence requests are simply agency reports that you generate in TOPSpro Enterprise. So let's go over each evidence request. The first one is data integrity reports, or DIR, for current and prior year. This is a TE report that shows a snapshot of all adult education students enrolled in your Adult Ed programs. The DIR contains information including, but not limited to, demographic data, pre post-test pair data, Integrated Education and Training or IET data, and so on.

The second evidence request is Federal Table 4, current and prior year. This is also a TE-generated report which reflects Measurable Skills Gains by entry level, or MSG by entry level. It includes information such as percentage of students achieving measurable skills gains, as well as total number of periods of participation per program. Federal Table 5, current and prior year-- this is a TE-generated report called Core Follow-up Achievement Report. It contains reportable outcomes, including employment, earnings, and post-secondary attainment.

The next evidence request is FERPA policy. Now, FERPA, as you probably already know, stands for Family Educational Rights and Privacy Act. FERPA is a federal law that protects the privacy of personal information in student education records.

Now, the law applies to all educational organizations that receive funds under an applicable program of the United States Department of Education. Providing the WIOA Title II AEFLA services in California entails program staff obtaining personal information or records from individuals applying for or receiving the WIOA Title II AEFLA services. Any personal information contained in these education records is protected under FERPA.

Social Security Consent Form-- FERPA generally requires an adult student's written consent for disclosure of Personally Identifiable Information, or PII, contained in education records, unless a specific exception applies. The CDE requests that any organization providing the WIOA Title II AEFLA services to adults in California ask for the voluntary participation in the provision of Social Security Numbers in the execution of a voluntary written consent, allowing the CDE to share participant data, so that the CDE can obtain employment and wage information required for performance reporting requirements under WIOA. So for this particular evidence request, we are asking you to upload a copy of your current Social Security Consent Form.

High school diplomas awarded, current and prior three years-- here, we are asking for evidence listing students that have been awarded their High School Diplomas, or HSD, in the current and prior three years. Include the date each student has completed their high school diploma. And please do not include students that have completed their HSE, or High School Equivalency. So what we're looking for here is a list of students who have received and have been awarded their High School Diplomas, not HSE or High School Equivalency.

The next evidence request is local assessment policy. We review your local assessment policy to make sure it is current and not missing the six required components. These six components are number one, training and dissemination of local guidelines for implementing the California assessment policy, two, initial orientation and placement into program and instructional level, three, progress testing-- this is your pre-test and post-test-- four, use of test administration manuals, five, training requirements for administering standardized assessments, and six, test security agreements.

Now, in an on-site review, we may ask you specific questions about your assessment policy and procedures. Also, if your remote testing agreement is not included in your local assessment policy, please upload a copy of your remote testing agreement.

Payment point summary, current and prior three years-- we compare your payment point summary with your HSD list or High School Diploma list, and also with your Section 243 co-enrollment list, which are both required evidence request in AE 10 and in this item, AE 03. If there is a discrepancy in numbers, we will require you to explain and/or perform an analysis of the data.

Persister reports, current and prior year-- This is a TE-generated report on educational gains and average attendance by Educational Functioning Level, or EFL. Again, this is a TE-generated report.

The bottom line and key points for AE 03 are these-- agencies who upload required evidence for this item successfully demonstrate that personnel are knowledgeable, and that the statistical and other data are properly maintained and secured for a period of three years. During an on-site review, we will also interview your CASAS coordinator and other key data staff.

David Stang: Under staff qualifications and professional development, please upload into CMT a list of all, not just federally funded, Adult Ed personnel and their respective assignments for the current and prior year. Duty statements for the current and prior year should be uploaded into CMT, as well, including statements for administrators and staff. Duty statements can include a range of evidence.

Ideally, we're looking for job descriptions or statements for each employee, describing responsibilities and activities. In some instances, however, job descriptions for teachers-- for example, in certain program areas-- may meet minimum compliance. ESL instructors under the category of beginning ESL may all have similar duty statements. And in some cases, depending on the description, that may be appropriate.

Professional development records to be uploaded into CMT are agendas, calendars, certificates, sign-in sheets, minutes, training materials, et cetera. Examples of the current and prior year should be included. Staff credentials-- the final evidence request for AE 04 should include a sortable spreadsheet of all certificated staff, displaying credentials and full staff name, including full middle name.

Credentials should be commensurate with current and prior year assignments. That is, teachers with single and multiple subject credentials at the K-12 level can teach in any subject area. Instructors at the community college obviously have a different set of minimum qualifications they must follow. But community colleges also must upload documentation or evidence the agency uses to determine candidate eligibility that meets minimum compliance.

Adult Ed teachers in the K-12 system with designated subject credentials must be working in those designated areas. Agencies that fall into the category of Community Business Organizations, or CBOs, must also provide some kind of documentation that addresses staff qualifications. Your agency may not require a specific teaching credential, but it should have some kind of written process, something in place that explicitly speaks to qualified staff and quality instruction in order to meet minimum compliance.

Abby Medina Lewis: We are halfway through our Adult Education Program Instrument. Now we have AE 05 needs assessment. For item AE 05, needs assessment, we are asking for a total of five evidence requests as follows-- AEFLA Demographic Reports, Federal Tables 1, 2, and 3, current and prior year.

Federal Table 1 is a TE report showing participants by Educational Functioning Level or EFL. Federal Table 2 is a report showing participants by age, ethnicity, and sex. Federal Table 3 is a TE report showing participants by program type and age. So for this evidence request, you are uploading three different Federal Tables for the current and prior year.

Community demographics report, current year-- examples of this could be US Census Bureau report or other statistically valid and reliable reports. English literacy and civics education needs assessment, current and prior year-- please upload a copy of your EL Civics Needs Assessment Survey that you administer to your students. Note that current and prior year is required here. If your current year survey is exactly the same as the last years, please indicate that in a comment in CMT so that the reviewer is aware that there is no change to your survey form.

Civics Objectives and Additional Assessment Plans, also known to all of you as COAAPs, current and prior year-- this also only applies to agencies funded for EL Civics. Basically, this is a printout of your selected COAAPs that CASAS has approved. Continuous Improvement Plan, current and prior year-- this is what we also refer to as the CIP plan.

As you know, the CIP is a new deliverable that consolidated the former three required deliverables-- PD plan, technology and distance learning plan, and IELC plan. For this evidence request, you are required to upload your approved CIP for current and prior year. The key point really here for needs assessment, AE 05, is that we are verifying whether your programs and services are consistent with the demographics in your area, and that you are responsive to regional needs.

AE 06, serving individuals with disabilities-- there is only one evidence request under this item, and that is the Americans with Disabilities Act, Individuals with Disabilities Education Act policy. Simply uploading your district non-discrimination policy will not meet minimum compliance for this item. To meet minimum compliance for this item, we are asking you to upload a copy of your agency's written policy to accommodate students and staff with disabilities.

Now, in the past, we've seen ADA policy that accommodates students, but we didn't get an upload for policy that accommodates staff with disabilities. So here, it is important that you upload written policy for accommodation for students and staff with disabilities, whether that's one document or two separate documents. We need to be able to see both.

Additionally, what procedures have your district or agency adopted to ensure students with IEPs or Individual Educational Plans and Section 504 plans have equitable access to your programs, activities, and transitional services? In the past, we've also received questions on what about if we're also receiving CAEP funds, and we have programs for adults with disabilities? Can we also upload a copy of our policy for programs for adults with disabilities in 504 or IEPs? Those are also examples of acceptable documentation.

David Stang: Each of the evidence requests under AE 07, intensity, duration, and flexible scheduling, require you to upload documentation for the current and prior year. Under this item, reviewers are looking for appropriate intensity, rigor, duration of time spent in the classroom, and flexibility on the part of the agency to provide services based on student need. It may be a concern, for example, or even a finding if services aren't offered at a time when students can attend class.

Course outlines would ideally be created at the district or academic program level. But outlines developed at the Adult Ed program level can meet minimum compliance. Be sure to consult with your reviewer. Sure, he may not ask you to upload every single outline, but rather a few examples from each program area simply to demonstrate compliance. Courses approved by the local board should indicate the board is aware some recognition of the Adult Ed program and the classes services are being provided.

Under AE 08, the distinction between sample curriculum and sample instructional materials, of course, is that curriculum is the overarching or umbrella materials, the program used to guide instruction. Any licenses the agency has purchased, textbook or publisher agreements which indicate the curriculum being used throughout the program, should be uploaded into CMT. Sample instructional materials are individual lesson plans, worksheets appropriate to the level of instruction, learning opportunities for adults, and not merely materials that would be used in a second- or third-grade classroom for elementary school students. Abby.

Abby Medina Lewis: Thank you, David. The next item is AE 09, effective use of technology and distance learning. Basically for this item, we are looking for evidence that technology is being used in the classroom. Note that the class schedule catalog under AE 07 and equipment inventory under AE 02 are shared evidence under this item.

Federal Table 5A, current and prior year- this evidence request applies only to agencies with distance learning programs. If your agency does not have a distance learning program, indicate that in a comment in CMT.

And finally, we are on AE 10, which is the last item in the Adult Education Program Instrument. AE 10 is Integrated Education and Training, or IET. This item only applies to agencies who are receiving WIOA Section 243 IELCE funds. If you do not have a Section 243 IELCE program, simply indicate that by posting a comment in CMT.

If your agency has an approved Section 243 IELCE program, then you are required to submit the following evidence. The first one is Program 243 Civics Objectives and Additional Assessment Plans, or COAAPs, current and prior year. This is your council's approved 243 COAAPs.

Evidence of co-enrollment, current and prior year-- this is a list of ESL students co-enrolled in a 243 IET course. The list should identify the training program that the ESL student is co-enrolled in. And finally, Integrated English Literacy and Civics Education Report or IELCE report, current and prior year. This is a copy of your CASAS-approved IELCE report.

Please make sure that you upload the version the CASAS specialist has approved, not the version that you submitted and has not yet been approved. Again, here we are looking for the IELCE report for the current and prior year that CASAS has approved. So that's it for our program instrument for AE 10. Now I'm going to turn you over to David so that he can share some common areas of concerns.

David Stang: Thanks, Abby. If we haven't already talked about this, we should take a minute to say, reviews are opportunities for the AEO consultants who are conducting the reviews and all of you to work together. We are not looking to create findings, take funding back from agencies. In fact, in some respects, the degree to which reviews are successful is an indication to the Feds that we are doing our due diligence to provide appropriate guidance, technical assistance, program development, strategies, et cetera.

That being said, there are a few areas in which to be particularly mindful, or common areas of concern. Be sure your time accounting documents reflect what is in your time accounting policies and procedures. If they don't match, that in itself could be a red flag to reviewers to take a closer look.

Also, take initiative. Reach out early to your reviewer to find out specifically what will meet minimum compliance. Always run purchases by your regional consultant when possible, to make sure they're allowable. Items such as equipment, furniture, graduation supplies, are often mistakenly thought of as allowable expenditures, when in reality they typically are not.

Assign someone knowledgeable about data and CASAS deliverables to work with you on the review. Obviously, there is a close relationship to data and federal funding, particularly in a pay for performance system like California. Make sure the list of high school diplomas you upload matches your payment point summary. Likewise, if you receive 243 funding, make sure you have a system in place to verify student co-enrollment, particularly in areas that relate to the amount of federal funding your area receives. Reviewers are looking to confirm what you report to us, and by extension, by what we report to the Feds.

Credentials-- be sure teachers are teaching in the area of their respective credentials, and that they're up to date. With respect to integrated education and training, again, be proactive. Reach out to your reviewer to learn exactly how to meet minimum compliance, and be sure your agency is able to verify co-enrollment of ESL students enrolled in workforce prep, workforce training.

OK, so what is a finding? In FPM, a finding is a three-part statement about requirements and program items which the LEA must meet, evidence which indicates the LEA is not meeting the requirements, and the steps the LEA must take to show that it subsequently is meeting the requirements. The official report of each FPM is the notification of findings.

Findings are rarely bad things, although we understand the negative perception and anxiety that often accompany them. Findings usually are opportunities to improve or shore up processes and/or systems already in place. In some instances, they are even notification to leadership that additional support or resources are needed. In any case, your reviewer has a responsibility to work with you to resolve any findings that come up in a review.

The resolution period includes an initial 45-day window in which all findings are to be resolved. Agencies must follow explicit instructions for how to resolve findings by the 45-day deadline. If resolution cannot be met during that time, agencies must request an extension with a reasonable justification and the expected length of time required to meet minimum compliance. In rare instances, up to 225 days may be required to resolve all findings. Resolutions-- resolution agreements may not go beyond 225 days.

Abby Medina Lewis: Now we would like to give you some tips for a successful review. The first one is communicate and coordinate with your district review coordinator if you are an LEA. If you are a non-LEA, make sure you communicate and coordinate with other individuals in your team who share a responsibility in the FPM.

Upload evidence into CMT early. You don't have to wait to upload until the 30-day evidence upload deadline that we discussed in the beginning of this session. You can start uploading evidence in CMT before the 30 days.

Again, in order for the reviewer to see your documents, they need to be certified. Make sure you certify the correct evidence and the evidence that you want the CDE reviewer to see. Document communication with a reviewer via CMT instead of email.

We understand that at times, email and/or phone communication may be warranted. The reviewer will also include email and phone communication in CMT. So as much as possible, you want to communicate and document your communication via CMT.

And in cases where you communicate by email, your reviewer will memorialize that in CMT, and it will be inputted in CMT, also. Finally, if you know you will need more time beyond the 45-day resolution period that David talked about to resolve the findings, please let your reviewer know as soon as possible. Just be proactive.

Here, we have provided you with resources and links that will help you prepare for your review. So take a few minutes to click on those links in order for you to see the content. If you don't know who your Adult Education Regional Consultant is, which will likely be your reviewer, you can click on the Resources that says 2022/23 Adult Education Office Regional Consultants to see your consultants and their contact information.

So this is the end of our session. On behalf of the CDE Adult Education Office, David and I thank you for reviewing and viewing this recorded training. Please remember to register for part 2 of the Adult Education Program Instrument training, which is the live question and answer webinar session scheduled on August 25 from 10:00 AM to noon. We look forward to answering questions you have in the Q&A webinar.

Thank you again. Take care. Be well and be safe.