Table of Contents

From AI Assistance to Independent Mastery

Designing an AI Learning Coach for College Mathematics Readiness

Project Introduction

Generative AI is rapidly changing how students learn mathematics. Students can now ask systems such as ChatGPT or Gemini to explain a concept, demonstrate a solution, identify an error, generate practice problems, or guide them through a difficult calculation. At the same time, established educational platforms such as ALEKS, WebAssign, Connect, MyLab, and WileyPLUS provide structured practice, assessment, and—in some cases—adaptive learning based on students’ demonstrated knowledge.

These technologies can be viewed as several stages in the evolution of digital mathematics learning.

Generation 1: Online Homework and Automatic Assessment

\[
\text{Problem}
\rightarrow
\text{Student Response}
\rightarrow
\text{Automatic Grading}
\rightarrow
\text{Feedback}
\]

Systems such as WebAssign made large-scale online mathematics practice and assessment practical. In this model, instructors largely determine what students study, while the system presents problems, evaluates responses, and provides feedback.

Generation 2: Adaptive Learning

\[
\text{Diagnose}
\rightarrow
\text{Estimate Knowledge}
\rightarrow
\text{Select Appropriate Material}
\rightarrow
\text{Practice}
\rightarrow
\text{Reassess}
\]

Adaptive systems such as ALEKS add individualized diagnosis and personalized learning paths. The system increasingly determines what each student is ready to learn, which prerequisite knowledge may be missing, and where additional practice is needed.

Generation 3: Conversational AI Learning

\[
\text{Student Struggles}
\rightarrow
\text{AI Conversation}
\rightarrow
\text{Explanation or Hint}
\rightarrow
\text{Continued Practice}
\]

Generative AI adds a powerful new capability: flexible, individualized conversation. A student can ask follow-up questions, request an alternative explanation, obtain hints, explore different solution methods, or ask the AI to explain a particular mistake.

This creates tremendous educational opportunities. However, it also introduces a fundamental problem:

A student being able to solve a problem with AI assistance is not the same as the student having mastered the mathematics.

A student may successfully complete homework while relying heavily on hints, explanations, worked examples, or AI-generated solutions. The completed work may therefore provide limited evidence about what the student can subsequently do independently.

The purpose of this project is not simply to develop another AI mathematics tutor. Instead, it will investigate how an AI Learning Coach can help students move systematically from assisted learning toward durable, independent mathematical mastery.

The Proposed Fourth Stage: AI That Builds Independence

The project explores a possible next stage in digital mathematics learning:

\[
\boxed{\text{Diagnose}}
\rightarrow
\boxed{\text{Practice}}
\rightarrow
\boxed{\text{Assist}}
\rightarrow
\boxed{\text{Diagnose Errors}}
\rightarrow
\boxed{\text{Fade Assistance}}
\rightarrow
\boxed{\text{Independent Practice}}
\rightarrow
\boxed{\text{Assess Mastery}}
\rightarrow
\boxed{\text{Check Retention and Transfer}}
\]

The AI coach would therefore have an unusual long-term objective:

Make its own assistance progressively less necessary.

When students struggle, the system should not automatically respond by providing more explanation. It should first try to determine why the student is struggling and then provide the least amount of assistance likely to support productive progress.

For example, an incorrect response might reflect:

a careless or execution error;
a notation or representation error;
a procedural error;
a conceptual misconception;
a missing prerequisite skill; or
difficulty recognizing which strategy should be used.

These situations call for different responses.

A simple arithmetic or sign error might justify asking the student to check the work again. A procedural error might require a small hint. A conceptual misconception might require explanation or questioning. A prerequisite gap might require temporarily leaving the current problem, repairing the earlier skill, and then returning to the original task.

The interaction can therefore be represented as:

\[
\text{Incorrect Response}
\rightarrow
\text{Diagnose Likely Error}
\rightarrow
\text{Select Intervention}
\rightarrow
\text{Student Retry}
\rightarrow
\text{Reassess}
\]

This leads to an important design question:

What is the least amount of assistance that will help the student make productive progress?

As competence increases, assistance should gradually decrease. Students would move from substantial coaching toward smaller hints, hints only when requested, and eventually AI-free problem solving.

Success during assisted practice would indicate progress, but independent performance would provide the stronger evidence of mastery.

From Assisted Performance to Durable Mastery

The project therefore needs a broader definition of mastery than simply answering several problems correctly.

An initial conceptual model is

\[
M=f(A,I,R,T),
\]

where

\[
A=\text{Accuracy},
\]

\[
I=\text{Independence},
\]

\[
R=\text{Retention},
\]

and

\[
T=\text{Transfer}.
\]

These dimensions ask four different questions.

Accuracy: Can the student solve the problem correctly?

Independence: Can the student solve it without AI assistance?

Retention: Can the student still demonstrate the skill after time has passed?

Transfer: Can the student apply the skill in a new or less familiar context?

The first prototype does not need to combine these variables into a sophisticated numerical mastery score. They can initially be tracked separately.

This distinction is important because short-term assisted success may not imply durable learning. A student might perform well immediately after receiving an explanation but struggle with the same skill several days later. Similarly, a student might solve familiar practice problems successfully but fail to recognize when the same mathematical idea is needed in a calculus problem.

The goal is therefore not merely successful practice. It is increasingly independent, retained, and transferable competence.

Initial Focus: Precalculus Readiness for College Mathematics

The first implementation will focus on foundational precalculus skills needed for success in college mathematics, particularly calculus.

Students can reach college calculus while retaining significant gaps in algebra, functions, trigonometry, arithmetic fluency, and related prerequisite skills. These weaknesses may interfere with calculus even when students understand the new calculus concept being taught.

Requiring such students to repeat an entire algebra or precalculus course may be unnecessary. Instead, an adaptive learning coach could identify specific prerequisite weaknesses and provide short, targeted remediation.

For example, consider a student struggling with a difference quotient who has difficulty expanding

\[
(x+h)^2.
\]

The immediate task appears to concern calculus, but the underlying difficulty may be algebraic.

Instead of simply completing the expansion for the student, the system could:

\[
\text{Detect Difficulty}
\rightarrow
\text{Identify Algebraic Prerequisite}
\rightarrow
\text{Provide Targeted Coaching}
\rightarrow
\text{Fade Assistance}
\rightarrow
\text{Verify Independent Algebra Skill}
\rightarrow
\text{Return to Difference Quotient}
\]

The student could then encounter a new calculus problem requiring the repaired algebraic skill. Successful use in that new context would provide evidence not only of retention but also of transfer.

This suggests a broader goal:

Provide the right prerequisite mathematics, to the right student, at the right time, with the right amount of assistance—and then determine whether the student can perform independently.

Learning Mode and Mastery Mode

One possible design implication is to make the distinction between learning and demonstrating mastery visible to students.

Learning Mode

During learning, AI assistance should be welcomed.

Students might:

ask questions;
receive hints;
request explanations;
explore alternative methods;
review prerequisite concepts;
examine related examples; and
receive increasingly explicit help when necessary.

The purpose is learning, not proving independence.

Mastery Mode

At appropriate points, students would solve new problems with little or no AI assistance.

The purpose changes from

\[
\text{Help me learn this}
\]

\[
\text{Can I now do this myself?}
\]

Separating these two modes may help resolve an important tension surrounding AI in education. AI assistance does not necessarily have to be prohibited during learning simply because independent competence is ultimately required.

Instead:

\[
\boxed{\text{Use AI generously for learning}}
\qquad
\text{but}
\qquad
\boxed{\text{verify mastery independently}}.
\]

A Human–AI Interaction Problem

This is fundamentally an HCI problem as well as a mathematics-education problem.

The important design question is not merely what the AI knows, but how and when the AI should interact with the learner.

The project will investigate questions such as:

When should AI provide an explanation, and when should it ask the student a question?
How should the system diagnose the likely cause of an incorrect response?
How much assistance should follow different types of errors?
When should the system revisit a prerequisite skill?
How can the system distinguish productive struggle from unproductive frustration?
When should hints become more explicit?
When should assistance begin to fade?
Should students be allowed to request more help after assistance has begun to fade?
How should students understand the difference between AI-assisted proficiency and independent mastery?
How should progress, AI dependence, retention, and mastery be visualized?
How can AI encourage students to think rather than simply obtain answers?
How can students retain agency while benefiting from adaptive guidance?
Can repaired prerequisite knowledge transfer to authentic college mathematics problems?

The project therefore combines mathematics education, learning science, adaptive learning, generative AI, and human–computer interaction.

Evaluating More Than Usability

Because this is an HCI project involving learning, evaluation should occur at several levels.

1. Usability

Can students understand and successfully interact with the system?

Possible questions include:

Is the interface clear?
Do students understand the assistance levels?
Do they understand why assistance is being reduced?
Can they distinguish Learning Mode from Mastery Mode?
Does fading support feel reasonable rather than arbitrary?

2. Learning Behavior

Does the design change how students interact with AI and approach mathematical problems?

Possible measures include:

\[
\text{Hint Consumption Rate},
\]

\[
\text{Attempts Before Requesting Help},
\]

\[
\text{Level of Assistance Required},
\]

\[
\text{Time on Task},
\]

and

\[
\text{Independent Problem-Solving Rate}.
\]

These measures may reveal whether students are gradually becoming less dependent on AI assistance.

3. Learning Outcomes

Ultimately, the project should ask whether students actually learn.

Relevant outcomes correspond to the mastery model:

\[
A,\qquad I,\qquad R,\qquad T.
\]

An immediate independent post-test can measure accuracy and independence. A short delayed assessment approximately one to two weeks later can provide preliminary evidence about retention. New problems or problems embedded in a calculus context can provide evidence about transfer.

This distinction is important:

A system can be easy and enjoyable to use without necessarily producing durable learning.

The project therefore seeks evidence about both the quality of the human–AI interaction and the learning that results from that interaction.

A Small Comparative Pilot

Once the interaction design and prototype have been tested for basic usability, the project may include a small comparative pilot.

One possible comparison is:

\[
\boxed{\text{Conventional AI Assistance}}
\qquad \text{vs.} \qquad
\boxed{\text{Mastery-Oriented AI Coaching}}
\]

In the first condition, students could receive conventional ChatGPT-style assistance.

In the second, students would use the proposed learning coach with structured hints, error-sensitive interventions, graduated assistance, and deliberate fading.

Both groups would eventually complete comparable problems independently.

The central question would not simply be:

Did students successfully complete the learning activity?

Instead, it would be:

After receiving AI-supported instruction, what can students solve independently?

A small pilot would not be expected to establish definitive causal evidence. Its purpose would be to generate preliminary evidence, identify promising behavioral and learning measures, expose weaknesses in the interaction model, and guide later research.

Beyond the Initial Learning Coach

The initial HCI project should remain deliberately focused. It should not attempt to build a complete learning-management system or solve every problem associated with AI and mathematics education.

However, the project fits within a larger vision containing three connected components.

1. AI-Assisted Learning

Students use AI productively to understand concepts, diagnose errors, receive individualized explanations, practice mathematics, and repair prerequisite weaknesses.

2. Independent Mastery and Assessment

Students periodically demonstrate what they can do without AI assistance. Immediate mastery checks can be supplemented by later retention and transfer assessments.

This becomes increasingly important when ordinary homework completion can no longer be assumed to represent independent student performance.

3. AI-Assisted Grading and Personalized Feedback

AI could eventually help evaluate mathematical reasoning and written solutions, apply instructor-defined rubrics, identify errors, recommend partial credit, and generate individualized formative feedback.

Human instructors would retain oversight, particularly for ambiguous, unusual, or low-confidence cases.

Ultimately, these components could form a continuous learning cycle:

\[
\boxed{\text{Learn with AI}}
\rightarrow
\boxed{\text{Demonstrate without AI}}
\rightarrow
\boxed{\text{AI-Assisted Evaluation}}
\rightarrow
\boxed{\text{Personalized Feedback}}
\rightarrow
\boxed{\text{Targeted Relearning}}
\]

The first project, however, will concentrate primarily on the critical transition from the first stage to the second:

How can an AI Learning Coach help students move from AI-assisted learning to verified, durable, and transferable independent mathematical mastery?

Project Goal

The goal of this project is not to determine whether students should or should not use AI. AI will increasingly be part of education, work, and everyday problem solving.

Instead, the project starts from a different premise:

Students should learn how to use AI effectively while also developing the knowledge and skills necessary to perform independently when independent competence matters.

A successful AI Learning Coach would therefore not seek to maximize AI use, minimize AI use, or simply maximize the number of correctly completed problems.

It would seek to maximize

\[
\boxed{
\text{Learning}
+
\text{Independence}
+
\text{Retention}
+
\text{Transfer}
}.
\]

This leads to the project’s central design principle:

The best AI learning assistance may be assistance that gradually teaches the learner not to need it.

The project will use HCI research, iterative prototyping, and preliminary empirical evaluation to investigate how this principle can be translated into an effective learning experience, beginning with precalculus readiness for college mathematics and potentially extending later to other areas of mathematics and AI-supported education.