CAROLYN ZACHRY: Welcome to the federal program monitoring training for the Adult Education Office. My name is Dr. Carolyn Zachary, and I am the state director and adult education administrator with the Adult Education Office at the CBE. 

David Stang and Abby Medina Lewis will be providing this webinar and the overview of our adult education instrument and all of the requirements related to your upcoming FPM. Please know that the FPM process while rooted while in compliance, will also give you the opportunity to find ways to improve your program for adult learners. Following the webinar, there will be the opportunity to have a live Q&A session with David and Abby. This will be set at a date in the future. Please enjoy the webinar. 

DAVID STANG: Thank you, Carolyn, for getting us started with another year. Hello, my name is David Stang. My colleague, Abby Medina Lewis, and I would like to welcome you and thank you for attending the adult education instrument virtual training for 2023, '24. 

For those of you who don't know us, Abby and I are education program consultants in the Adult Education Office or AEO at the California Department of Education. This training will focus on Federal Program Monitoring or FPMs and more specifically, the adult education instrument for the upcoming year. 

Speaking of the instrument, you should all have a copy of this year's instrument with you today. We will refer to it often, and you will want to follow along and take notes. If you do not have a copy already, please pause the presentation and download or print a copy. You can find the instrument at the first link of the resources and links slide at the end of the presentation. Abby, shall we get started? 

ABBY MEDINA LEWIS: Sure thing. Thank you for the introduction, David. Hello, everyone. This is Abby Medina Lewis. The purpose of today's training is to define compliance monitoring, what it is, and what it's not. 

This session will also provide you with an overview and understanding of the CDEs adult education program instrument, which is a tool used to ensure agencies receiving WIOA Title II AEFLA funds are in compliance with federal and/or applicable state law. And lastly, this session will provide clear expectations on the specific evidence required for each item on the adult education program instrument. 

Today's agenda includes federal program monitoring as stipulated by the Workforce Innovation and Opportunity Act, Adult Education and Family Literacy Act, which is also known to you as WIOA Title II AEFLA program. We will look at the adult education program instrument items and review evidence requests that you will be expected to provide and upload into the CDE Monitoring Tool or CMT. 

Just as a quick side note here, CDEs monitoring tool training is a separate training and will not be addressed in this session. The FPM office will provide the training on CMT. And details regarding CMT training will be shared via email soon. In the meantime, we provided you with a link to the CMT web page in the Resources page of this PowerPoint slide, which we will show you at the end of this session. 

David and I will also hold an interactive live three-hour question and answer webinar on August 8 from 9:00 AM to noon. Agencies scheduled for review this year must view this pre-recording in its entirety prior to participating in the live Q&A session on August 8. Only those participants that have viewed this pre-recorded session will be allowed to participate in the live Q&A webinar. Additionally, you will be required to register to participate in the Q&A session. We will email you the link to register for the Q&A webinar a week before the session. 

Finally, in the live Q&A webinar, you will also have an opportunity to hear from two agencies that were reviewed last year. Guest presenters will share their FPM experience as well as share strategies for a successful review. You will also have an opportunity to ask them questions. I want to emphasize again that viewing this recording and attending the live Q&A webinar on August 8 is required if you are scheduled for an FPM this year. 

DAVID STANG: OK, so what is federal program monitoring? The WIOA Title II AEFLA grant requires the CDE to conduct onsite and online monitoring of all adult education providers. Any and all adult education federal grant recipients will be selected for review at some point. On-site reviews are typically more comprehensive in nature. They are conducted at the adult education agency, include a review of documents or evidence requests, observations of the program offerings and classrooms, and interviews with key personnel. 

Online reviews are similar but do not include much of what occurs in the review process at an actual site or location. The review of documents is the same in both processes. In the past and because of COVID-19, most reviews during the last few years were conducted via an online or telemonitoring process. As of last year however, agencies are being reviewed online and onsite again. 

Please note, some offices and staff at CDE continue to work remotely. This means that even if you are scheduled for an on site review, one or more reviewers may decide to conduct the review online. Your reviewer will let you know when this happens. The purpose of monitoring reviews is to ensure agencies are in compliance with federal and where applicable, state laws. Current monitoring cycle is for this program year, 2023, '24. Abby. 

ABBY MEDINA LEWIS: But what is compliance monitoring? Compliance monitoring is simply an overall determination of whether agencies receiving federal funds are meeting statutory program and fiscal requirements for certain programs. It is CDE's fulfillment of regulations to ensure that state and federally-funded agencies meet minimum compliance. It is also an opportunity to examine and improve your agency policies and procedures. This could include but not limited to your assessment policy, no fees policy, time accounting, and other programmatic and fiscal policies. 

Compliance monitoring is not an audit. We are not auditors. We are program reviewers. It is not a gotcha opportunity to take funding back from your program. 

DAVID STANG: All programs, whether they be adults before and after school or fiscal monitoring, develop program instruments to guide monitoring reviews. And within those instruments are categories or items that the reviewer looks at during each review. The adult education instrument has 10 items or categories of which Abby and I are going to review with you today. Abby, will you get us started? 

ABBY MEDINA LEWIS: Of course. Thanks, David. The 2023, '24 instrument is organized into 10 different categories or items. Each item has a respective list of evidence requests that must be provided by agencies to meet federal requirements. 

These 10 adopted items are AE 01, collaboration alignment and support services, AE 02, financial accountability, AE 03, data collection and program effectiveness, AE 04, staff qualifications and professional development, AE 05, needs assessment, AE 06, serving individuals with disabilities, AE 07, intensity, duration, and flexible scheduling, AE 08, evidence-based instructional practices and reading instruction, AE 09, effective use of technology and distance learning, and finally AE 10, integrated education and training. 

These 10 instrument items may look and sound familiar to you because if you recall when you applied for the grant, these are the same as the federally imposed considerations that you are required to respond to in the WIOA Request For Application or RFA. 

So what are evidence requests? Evidence requests are examples of documents that agencies upload into the CDE monitoring tool, CMT, to demonstrate compliance. All evidence requests must be uploaded into CMT 30 days prior to the first day of a review. So, for example, if the first day of your review is scheduled on November 1, then you are required to upload and certify all evidence requests by October 1. 

It's important to get all your evidence into CMT 30 calendar days prior to the first day of your review to allow the CDE program reviewer just enough time to review the evidence and provide you with comments on each instrument item. The CDE program reviewer will provide updates for each item no later than 15 calendar days prior to the first day of the review. If you are unsure about a particular evidence request, contact your CDE adult education FPM reviewer and ask for clarification. 

Typically, your region CDE adult education consultant is also the assigned reviewer for your FPM. However, there are some instances where a different reviewer is assigned to your review, such as if there are scheduling conflicts. Now we will begin our deep dive of the 10 adult Ed instrument items that I shared with you in your previous slides. 

DAVID STANG: Before we look at the first item, would you please look at page one of the instrument? I'd like to bring your attention to the initial description or legal guidance section that you will see for each item. Item one or AE 01, collaboration alignment and support services, includes all of the federal guidelines the Adult Education Office uses to guide the review for this item. We highly encourage you to read each guidance section for all of the items to, one, provide you with an overview of the item, that is, what the item is really about. Collaboration, alignment, support services, for example, you should expect to see guidance in these areas. 

The first guideline, 1.0, for example, reads, "Eligible provider demonstrates alignment between proposed activities and services and the strategy and goals of the local plan as well as the activities and services of the one-stop partners." Guideline 1.1 reads, "Eligible providers activities coordinate with other available education, training, and social service resources in the community." 

Secondly, the AE0 included each guideline or federal regulation in the initial section of each item as a foundation for each evidence request that is required within the adult education instrument. This is what we are looking for in this section. And these are the areas we are required to monitor for this item. 

It makes sense, for example, that under AE 01, we will see collaboration with other partners and providers, that we will see alignment with the local plan and support services for students. In addition, if there are any findings during the course of the review, these guidelines are what we use to form the legal basis of each finding. 

AE 01, collaboration, alignment, and support services. With slide 14, we're going to take a closer look at actual evidence requests. Each item includes a number of documents or evidence requests agencies are required to upload into CMT, the CDE monitoring tool. 

In many instances, we are asking for a very specific piece of evidence. The purpose of requiring evidence requests is to ensure recipients of federal funding meet minimum compliance. To do so, we ask that you upload the requested documents or in some cases, the equivalent evidence to meet compliance. 

If you look on page two of the adult ED instrument, the first evidence request you see is Memorandum of Understanding or MOU. The description for MOU reads, "MOUs or formal, informal agreements that demonstrate current collaborations or partnerships with the adult education programs." 

Agencies are required to upload at least one MOU or informal agreement with another provider, contractor, or agency that reflects the types of services being provided by each partner, that is, how are the agencies collaborating? On behalf of students, what does the partnership involve? Can agencies upload more than one MOU? Of course, I'd say two or three are ideal. In order to meet minimum requirements for this item, however, at least one MOU must be uploaded. 

The description for the next evidence request resource guide indicates evidence that demonstrates resources available to adult education students, resources such as child care, transportation, counseling and tutoring for the current and prior year. Agencies should have available to students a resource guide of some kind, a pamphlet, in some cases a catalog. Even a web page can be used to meet minimum compliance for this request. 

The point is, how do students know what services are being offered? What examples of support are available to them in the event they need child care, for example, a counselor or tutoring? If you also notice, this evidence request indicates agencies must upload a resource guide for the current and prior year. Some say current and prior three years. In some cases, if your review is scheduled early in the year, you may not have anything to upload for the current year. That's OK. 

Please be sure to clarify with your reviewer what if anything needs to be uploaded if you see current year and your review is scheduled within the first few months of the year. In this case, agencies should be able to upload a resource guide from the current year since ideally you would have that information available for students at the start of the year. But if for some reason it is still in development, your resource guide from the previous year would be sufficient. 

The next evidence request under secondary evidence request reads, "Supplemental evidence of collaboration." This is not a required request. In the event an agency is having trouble meeting minimum compliance, for example, an agency doesn't have an MOU or a resource guide available, the reviewer in some cases might ask for supplemental evidence of collaboration to help an agency meet minimum compliance. Having an MOU, for example, is a requirement. 

But in some cases, if an agency has both phase one and phase two of their umbrella MOU with the AJCC in place, a reviewer might be OK with an agency uploading supplemental evidence of collaboration if they don't have a current MOU in place. If you look at the description, examples of supplemental evidence could be recent meeting agendas, sign-in sheets, meeting minutes, et cetera. But only upload these documents if asked by your reviewer. 

This brings us to the final evidence request for AE 01. Description for this evidence request reads, "A current signed MOU with the local workforce development board that outlines the activities and services of each partner in the overarching AJCC delivery system to support the goals of the local plan." This request includes both phase one and phase two of the MOU. Some agreements may include both in one document. 

Typically, each phase is developed separately. But in either case, phase one should reflect all of the services being provided by each of the partners involved in the MOU as well as the name of each partner and actual signatures from the appropriate representatives at your agency. It is important that you locate these documents now and ensure that you have a complete copy of the MOU, which includes signatures from all partners. 

The process to request a copy from the LWGB during the course of a review can be time-consuming and can even lead to a finding if you are not able to upload this request before the end of the review. A little work on the front end goes a long way so that you are able to focus your attention on more important matters during the course of your review. Phase two should be an MOU with all of the cost-sharing agreements between all of the agencies. 

AE 02, financial accountability. Again, as we move to AE 02, please remember to read the initial guidance section for this item. The initial evidence request description for AE 02 reads, "Historical inventory list of all equipment purchased for $500 or more per education department general administrative regulations and/or California Ed code requirements and a record of last physical check of items." If no purchases were made, indicate as much by posting a comment in CMT. 

For this evidence request, you will also see under Item Instructions specific guidance related to equipment inventory outlined by the Adult Education Office. Provide percentages of Adult Education and Family Literacy Act funds use, serial numbers, resource codes, location of equipment, and items that were disposed of and how they were disposed, current and prior three years. 

If your agency has not purchased any equipment this year, be sure to indicate that in a comment. And if you have any questions about the format or how to organize the document, again, be sure to ask your reviewer. Also, be aware and anticipate potential issues. Your reviewer will cross reference any information your agency provided to the AE 0 regarding the purchase of equipment via expenditure claim reports, continuous improvement plans, or otherwise with what you upload into CMT. 

Let the reviewer know ahead of time if there are discrepancies so both the reviewer and the agency have time to troubleshoot if necessary. The description for general ledger is a general description used by all program offices at CDE, detailed general ledger for the specific resource codes being reviewed. The specific instructions pertaining to the Adult Education Office are to include specific resource codes for adult Ed. 3905, 3913, 3926, and 6391 are all examples of the resource codes we would expect to see in an agency general ledger. Ledgers from the current and prior three years should be uploaded into CMT. 

What reviewers are looking for primarily with this evidence request are, one, to confirm the expenditures which were reported to the CDE in the final ACR for each object and resource code and, two, ensure that federal expenditures are reasonable, necessary, and allowable under federal regulations. 

The description for in-kind and other non-federal expenditures requires agencies to explain or document how expenditures are calculated and included in the final Expenditure Claim Reports or ECRs. Again, what you upload into CMT should be reflected in each year's ECR. Your description can be as simple word document, which reflects your agency's actual in-kind expenditures and aggregate, and/or an explanation of what your in-kind expenditures are. 

Be sure to ask your reviewer if you're not quite sure what in-kind means or if you're not the one who submits the final ECR each year. At the end of your quarterly ECR, there is a definition of what other non-federal and/or in-kind contributions are under the page for non-federal sources of funds expended for WIOA Title II AEFLA grant-related activities. 

Agencies are not allowed to charge student fees for any program where federal funds are provided to agencies. Registration fees, fees for books, tuition are all examples of fees agencies are not allowed to charge. Agencies should include in their catalog or website, resource guide, somewhere clearly visible to students they are not being charged fees. 

Ideally, we encourage and would like to see an actual policy indicating students will not be charged fees in any federally sponsored adult Ed program. However, as long as an agency clearly communicates it is not charging fees, it should meet minimum compliance. Agency's fee policies for the current and prior year should be uploaded. 

Org charts should include the names and titles of staff and board members reflecting leadership at the district and adult education level. Board minutes only apply to community-based organizations, including charter schools. Two samples from each year for the current and prior year will meet minimum compliance. 

The description for staff funding report indicates that funding percentages for all adult education personnel should be included. There is a template or example on the compliance monitoring web page at CDE. Or simply ask your reviewer for an electronic copy. The report should be for the current and prior three years. Again, it's for all adult ED staff, not just AEFLA-funded positions. Reviewers look at this request over the three-year period to establish a trend to ensure state dollars are the primary source of funding for the overall compensation work performed by adult Ed personnel. 

Reviewers also cross-referenced the administrative and staff assignments evidence request to ensure all staff reflected in the evidence request are also included in the staff funding report. Time and effort policies and procedures are written policies for documenting time and effort of employees that work on federal programs. Your agency's policies should include specific practices for documenting actual hours worked as well as internal controls and employee training around accounting for time. 

The two most important components of this request are first to make sure your agency has a policy. If you don't have one now, get started on it right away. You may need board approval before the start of your review, which can take several months. And second, make sure what is reflected in your policy is what is actually taking place with respect to time accounting. Please work with your reviewer if you have more specific questions. 

Examples of time and effort records can include Personnel Activity reports or PAs, semiannual certifications, or other equivalent records. Documentation to support salaries and benefits charged to each program funding source under review should be provided. Again, work with your reviewer to determine the number of samples for each quarter to be uploaded into CMT. 

Supplemental evidence of financial accountability, this request was added to the instrument last year to provide both agencies and reviewers additional opportunities to meet compliance. Again, as Abby mentioned, the purpose of the review is not to afford an opportunity to take money back from agencies. It is the agency's responsibility to provide the appropriate evidence or documentation. It is your reviewer's responsibility to assist your agency with knowing what evidence meets minimum compliance. 

In the event reviewers need more information to meet compliance, they may ask you to upload additional documentation in this area. Examples of supplemental evidence are payroll records, position control reports, detailed ledgers for object codes 4,000 and 5,000. That does it for financial accountability. 

ABBY MEDINA LEWIS: AE 03, data collection and program effectiveness. The city requires all WIOA Title II AEFLA grantees to use the TOPSpro Enterprise or TE database system to collect and report adult learner demographics and program information. In addition, the WIOA title II AEFLA-funded agencies are required to comply with the National Reporting System or NRS for adult education requirements. 

TOPSpro uses the federal tables to report statewide aggregation totals to the federal government. For this item, AE 03 data collection and program effectiveness, we are verifying if your agency is meeting data accountability requirements in accordance with WIOA Title II AEFLA grant. There are nine evidence requests related to data collection and program effectiveness of which five evidence requests are simply agency reports that you generate in CASAS TE. So let's go over each evidence request here. 

The first one is Data Integrity Reports or DIR, current and prior year. This is a TE report that shows a snapshot of all adult education students enrolled in your adult Ed programs. The DIR contains information, including but not limited to demographic data, pre-post test pair data, integrated education and training or IET data and so on. 

Federal tables 4 current and prior year, this is also a TE-generated report which reflects measurable skill gains by entry level. It includes information such as percentage of students achieving measurable skill gains as well as total number of periods of participation per program. 

The next evidence request is federal table 5, current and prior year. This is a TE-generated report called primary indicators of performance. It contains reportable outcomes, including employment, earnings, and post-secondary attainment. 

The next evidence request is FERPA Policy, current year. FERPA is Family Educational Rights and Privacy Act. It is a federal law that protects the privacy of personal information in student education records. 

The law applies to all education organizations that receive funds under an applicable program of the United States Department of Education. Providing the WIOA Title II AEFLA services in California entails program staff obtaining personal information or records from individuals applying for or receiving the WIOA Title II AEFLA services. Any personal information contained in these education records is protected under the FERPA law. 

Social Security consent form, current year, FERPA generally requires an adult education written consent for disclosure of Personally Identifying Information or PII contained in education records unless a specific exception applies. The CDE requests that any organization providing the WIOA Title II services to adults in California ask for the voluntary participation in the provision of Social Security numbers and the execution of a voluntary written consent allowing the CDE to share participant data so that CDE can obtain employment and wage information required for performance reporting requirements in the WIOA. 

So for this particular evidence request, we are asking you to upload a copy of your current Social Security consent form. To meet minimum compliance, consent forms must include statement, including students who do not provide a Social Security number are still eligible to participate in the adult education program. 

The next evidence request is high school diplomas awarded, current and prior three years. Here we're asking for evidence listing students that have been awarded High School Diplomas or HSD in the current and prior three years. Include the date each student has completed their high school diploma. Please do not include students that have completed their High School Equivalency or HSE. 

Local assessment policy, we review your local assessment policy to make sure that it is current and not missing the six required components. These six components are training and dissemination of local guidelines for implementing the California Assessment Policy, initial orientation and placement into program and instructional level, progress testing, pre-test and post-test, use of test administration manuals, training requirements for administering standardized assessments, and finally test security agreements. 

In an on-site review, we may ask you specific questions about your assessment policy and procedures. Also, if your remote testing agreement is not included in your local assessment policy, please upload a copy of your remote testing agreement. Your local assessment policy must be updated annually and reflect the date last reviewed with your staff. 

The next evidence request is payment point summary report, current and prior three years. We compare your payment point summary with your high school diploma list and also with your section 243 co-enrollment list, which is a required evidence request in AE item 10. If there is a discrepancy in numbers, we will require you to explain and/or perform an analysis of the data. 

Our sister reports current and prior year, this is a TE report on educational gains and average attendance by Education Functioning Level or EFL. The bottom line and key point for AE 03 are these. Agencies who upload required evidence for this item successfully demonstrate that personnel are knowledgeable and that statistical and other data are properly maintained and secured for a period of three years. During an on-site visit, we also interview your CASAS coordinator and other key data staff. 

DAVID STANG: AE 04, staff qualifications and professional development. Under Staff Qualifications and Professional Development, please upload into CMT a list of all, not just federally-funded, adult Ed personnel and their respective assignments for the current and prior year. Duty statements for the current and prior year should be uploaded into CMT as well, including statements for administrators and staff. Duty statements can include a range of evidence. 

Ideally, we're looking for job descriptions or statements for each employee describing responsibilities and activities. In some instances however, job descriptions for teachers, for example, in certain program areas, may meet minimum compliance. 

ESL instructors under the category of beginning ESL may all have similar duty statements and in some cases, depending on the description, that may be appropriate. Professional development records to be uploaded into CMT are agendas, calendars, certificates, sign-in sheets, minutes, training materials, et cetera. Examples for the current and prior year should be included 

The final evidence request for AE 04, staff credentials, should include a sortable spreadsheet of all certificated staff displaying credentials and full staff name, including full middle name. Credentials should be commensurate with current and prior year assignments. That is, teachers with single and multiple subject credentials at the K-12 level can teach in any subject area. 

Instructors at the community college obviously have a different set of minimum qualifications they must follow. But community colleges also must upload the documentation or evidence the agency uses to determine candidate eligibility that meets minimum compliance. Adult Ed teachers in the K-12 system with designated subject credentials must be working in those areas. 

Agencies that fall under the category of community business organizations must also provide some kind of specific teaching credential. But it should have some kind of written process, something in place that explicitly speaks to qualified staff and quality instruction in order to meet minimum compliance. Your process should include minimum qualifications and be consistent with the duty statement for that position. 

ABBY MEDINA LEWIS: AE 05, needs assessment. For item AE 05 needs assessment, we are asking for a total of five evidence requests as follows, AEFLA demographics reports Federals table one, two, and three, current and prior year. Federal table one is a report showing participants by Educational Functioning Level or EFL. Federal table two is a TE report showing participants by age, ethnicity, and sex. Federal table three is a TE report showing participants by program type and age. 

The second evidence request is community demographics reports, current year. Examples of this could be US Census Bureau report or other statistically valid and reliable reports. English literacy and Civics education needs assessment, current and prior year. 

Upload a copy of your EL Civics needs assessment survey that you administer to your students. Note that current and prior year is required here. If your current year survey is exactly the same as the last year's survey, please indicate that in a comment in CMT so that the reviewer is aware that there is no change in your survey form. 

The next evidence request is Civics Objectives and Additional Assessment Plans, also known to most of you as COAAPs, current and prior year. This also only applies to agencies funded for EL Civics. Basically, this is a printout of your selected COAAPs that CASAS has approved. 

The final evidence request for this item is Continuous Improvement Plan or CIP, current and prior year. For this evidence request, you are required to upload your approved CIP for current and prior year. Again, please upload the version that has been reviewed and approved by the CASAS, OTAN, and CDE team reviewers. The key point for AE 05 needs assessment is that we are verifying whether your programs and services are consistent with the demographics in your area and that you are responsive to regional needs. 

AE 06, serving individuals with disabilities. There is only one evidence request under this item, and this is the Americans with Disabilities Act, ADA, Individuals with Disabilities Education Act, IDEA policy. Simply uploading your district non-discrimination policy will not meet minimum compliance for this item. 

To meet minimum compliance for this item, we are asking you to upload a copy of your agency's written policy to accommodate students and staff with disabilities. Again, I'm going to emphasize to accommodate students and staff with disabilities. Additionally, what procedures have your district or agency adopted to ensure students with IEPs or Individual Education Plans section 504 and/or SELPA, Special Education Local Plan Area plans, have equitable access to your programs, activities, and transitional services? 

DAVID STANG: AE 07, intensity, duration, and flexible scheduling. Each of the evidence requests under AE 07, intensity, duration, and flexible scheduling, require you to upload documentation for the current and prior year. Under this item, reviewers are looking for appropriate intensity, rigor, duration of time spent in the classroom, and flexibility on the part of the agency to provide services based on student need. 

It may be a concern, for example, or even a finding if services aren't offered at a time when students can attend classes. Course outlines would ideally be created at the district or academic program level. But outlines developed at the adult Ed program level can meet minimum compliance. 

Be sure to consult with your reviewer. She or he may not ask you to upload every single outline but rather a few examples from each program area simply to demonstrate compliance. Courses approved by the local board should indicate the board is aware, some recognition of the adult Ed program and the classes services being provided. For LEAs, court's approval letters are also acceptable documentation. 

AE 08, evidence-based instructional practices and reading instruction. Under AE 08, the distinction between sample curriculum and sample instructional materials of courses that curriculum is the overarching or umbrella materials the program used to guide instruction. Any licenses the agency has purchased, textbook, or publisher agreements which indicate the curriculum being used throughout the program should be uploaded into CMT. 

Sample instructional materials are individual lesson plans, worksheets appropriate to the level of instruction, learning opportunities for adults and not merely materials that would be used in a second or third grade classroom for elementary school students. 

ABBY MEDINA LEWIS: AE 09, effective use of technology and distance learning. So basically for AE 09, we are looking for evidence that technology is being used in the classroom. Note that class schedule/catalog under AE 07 and equipment inventory under AE 02 are shared evidence under this item. 

Federal table 4C, current and prior year. This evidence request is a TE report named measurable skill gains in distance learning programs. This applies only to agencies with distance learning programs. If your agency does not have a distance learning program, indicate that in a comment in CMT. 

Federal table 5A, current and prior year, this is a TE report named primary indicators of performance in distance learning programs. This evidence also applies only to agencies with distance learning programs. 

AE 10, integrated education and training. The final item is integrated education and training. This item only applies to agencies who are receiving WIOA section 243 IELCE funds. If you do not have a section 243 IELCE program, simply indicate this by posting a comment in CMT. If your agency has an approved section 243 IELCE program, then you are required to submit the following evidence. 

The first one is program 243 Civic Objectives and Additional Assessment Plans or COAAPs, current and prior year. This is your CASAS approved 243 COAAPs. Evidence of co-enrollment, current and prior year, this is a list of ESL students co-enrolled in a 243 IET course. The list should identify the training programs that the ESL student is enrolled in. 

Integrated English Literacy and Civics Education, IECLE report, current and prior year. This is a copy of your CASAS-approved IELCE report. Please make sure that you upload the version that CASAS specialist has approved. 

DAVID STANG: Key points, if we haven't already talked about this, we should take a minute to say reviews are opportunities for the AEO, the consultants who are conducting the reviews, and all of you to work together. We are not looking to create findings or take funding back from agencies. In fact, in some respects the degree to which reviews are successful is an indication to the feds that we are doing our due diligence to provide appropriate guidance, technical assistance, program development strategies, et cetera. 

That being said, there are a few areas in which to be particularly mindful. Be sure your time accounting documents reflect what is in your time accounting policies and procedures. If they don't match, that in itself could be a red flag to reviewers to take a closer look. Also, take initiative. Reach out early to your reviewers to find out specifically what will meet minimum compliance. 

Always run purchases by your regional consultant when possible to make sure they're allowable. Items such as equipment, furniture, graduation supplies are often mistakenly thought of as allowable expenditures when in reality they typically are not. Assign someone knowledgeable about data and deliverables to work with you on the review. 

Obviously, there is a close relationship to data and federal funding, particularly in a pay for performance system like California. Make sure the list of school diplomas you upload matches your payment point summary. Likewise, if you receive 243 funding, make sure you have a system in place to verify student co-enrollment, particularly in areas that relate to the amount of federal funding your agency receives. Reviewers are looking to confirm what you report to us and by extension what we report to the feds. 

Credentials, be sure teachers are teaching in the area of their respective credentials and that they're up to date. It's easy to say the district should have apprised me or the instructor should have informed me. But it is not a pleasant experience for anyone involved, especially teachers, when individual teacher must be removed from the classroom for not having proper credentials. Have a process in place. So teacher credentials are reviewed each year for validity and current status. 

With respect to integrated education and training, again, be proactive. Reach out to your reviewer to learn exactly how to meet minimum compliance. And be sure your agency is able to verify co-enrollment of ESL students and enrolled in workforce prep or workforce training. 

In FPM, a finding is a three-part statement about requirements and program items which the LEA must meet, evidence which indicates the LEA is not meeting the requirements and the steps the LEA must take to show that it subsequently is meeting those requirements. 

The official report of each FPM is the Notification of Findings or NOF. Findings are rarely bad things. Although we understand the negative perception and anxiety that often accompany them. Findings usually are opportunities to improve or shore up processes and/or systems already in place. In some instances, they are even notification to leadership that additional support or resources are needed. In any case, your reviewer has responsibility to work with you to resolve any findings that come up in a review. 

The resolution period includes an initial 45-day window in which all findings are to be resolved. Agencies must follow explicit instructions for how to resolve findings by the 45-day deadline. If resolution cannot be met during that time, agencies must request an extension with a reasonable justification and the expected length of time required to meet minimum compliance. In rare instances, up to 225 days may be required to resolve all findings. Resolution agreements may not go beyond 225 days. 

ABBY MEDINA LEWIS: OK, we are almost to the finish line. We would like to give you some tips for a successful review. The first one is communicate and coordinate with your district review coordinator if you are an LEA. If you are a non-LEA, make sure you communicate and coordinate with the other individuals in your team who share a responsibility in the FPM. 

Second tips for success is upload evidence into CMT early. We could not emphasize this enough. You don't have to wait to upload until the 30-day evidence upload deadline that we discussed in the beginning of this session. You can start uploading evidence into CMT before the 30 days. Again, in order for the reviewer to see your documents, they need to be certified. Make sure you certify the correct evidence and the evidence that you want the CDE reviewer to see. 

The third tip is document communication with your reviewer via CMT. Document communication with a reviewer via CMT instead of email. We understand that at times email and/or phone communication may be warranted. The reviewer will also include email and phone communication in CMT. Finally, if you know you will need more time beyond the 45-day reservation period to resolve findings, please let your reviewer know as soon as possible. 

In this last slide, we have provided you with the resources and links that will help you prepare for your review. The first is the link to this year's adult education program instrument. Please make sure that you print a copy of the 2023, '24 instrument for your reference in the August 8 Q&A webinar. 

The second link is where you can find the CDE's consultants assigned to your region. The third link is where you can find FAQs regarding federal program monitoring. And finally, the last link gives you information on the CDE's Monitoring Tool or CMT. 

We have reached the end of our session. On behalf of the CDE Adult Education Office, David and I thank you for viewing this recorded training. Please remember to register for part 2 of the adult education program instrument training, which is the live Q&A webinar scheduled on August 8 from 9:00 AM to noon. We look forward to answering questions that you have in the Q&A webinar. Thank you again, take care, and be well.