/
Thesis · 2024

Human with AI

The impact of AI English conversation interface types on user experience, focusing on the moderating effect of proficiency.

Mockup of the real-human conversation prototype

Impact

  • Demonstrated that AI interface types affect immersion, cognitive load, and satisfaction differently depending on learner proficiency, providing empirical evidence for adaptive AI learning design.
  • Identified a clear trade-off in human-like AI interfaces: higher immersion and satisfaction, but also a heavier psychological burden, especially for low-proficiency learners.
  • Revealed that simplified, low-burden interfaces are more effective for beginners, providing actionable UX insights for personalized AI education platforms.

Role

UX Researcher / Product Designer

Timeline

May – Aug 2024 (3 Months)

Team

1 Researcher

Tools

Figma · Illustrator · Photoshop · SPSS / Excel

Contribution

Led research and design on AI-driven conversation prototypes, focusing on user immersion, cognitive load, and learning satisfaction across levels of anthropomorphism.

  • Designed and tested four AI interface types (text, animal, avatar, human-like).
  • Conducted quantitative and qualitative analysis with 100 learners.
  • Created adaptive UI prototypes reflecting level-based insights.

01

Overview

Do different types of AI anthropomorphism affect users' learning outcomes?

This study analyzed how four types of conversational AI interfaces in English-speaking apps, voice/text-based, animal character-based, human character-based, and real human-based, influence learners' experience: immersion, cognitive load, and satisfaction.

The real-human interface produced the highest immersion and satisfaction but also raised users' psychological burden, while text- and voice-based interfaces offered a more stable experience for beginners. Designing the right level of anthropomorphism for a learner's proficiency and context is what improves both immersion and comfort in AI-assisted conversation learning.

02

Background

The rise of conversational AI in language learning

Conversational AI learning has drawn attention for enhancing efficiency and engagement beyond traditional methods. It offers learners a more immersive and sustainable experience that keeps them motivated.

Collage of video-call English conversation app screens

The debate on AI anthropomorphism

Proponents argue that anthropomorphism in AI enhances engagement and empathy, improving learning and service experiences. Opponents warn that human-like AI may raise ethical and credibility issues that lead to distrust.

Proponent

Ben Goertzel

Believes anthropomorphism can spark curiosity and emotional connection with AI.

Opponent

Dor Skuler

Warns about the risks of giving AI human-like forms and attributes.

03

Research Design

Building on prior research, the study analyzed how four types of anthropomorphism in English-speaking learning apps affect learner experience, with a particular focus on how the appropriateness of the dialogue type shapes engagement and satisfaction.

Independent variable

  1. AVoice / Text-based conversation
  2. BAnimal character-based conversation
  3. CHuman character-based conversation
  4. DReal human-based conversation

Moderating variable

Conversation proficiency

High / Low

Dependent variable

  1. 01Immersion
  2. 02Cognitive load
  3. 03Learning satisfaction

H1

Immersion, cognitive load, and learning satisfaction will differ by the anthropomorphism type of the AI interface.

H2

Conversation proficiency will moderate the relationship between interface type and learning experience.

04

Method

To support the survey findings, short in-depth interviews (under 20 minutes) were conducted with selected participants. High and low proficiency groups experienced identical AI conversation prototypes, and their immersion, emotional response, and satisfaction were compared across anthropomorphism types.

The four AI conversation prototypes used in the experiment
Experiment prototypes. From left: voice/text, animal character, human character, real human.
In-depth interview session
In-depth interview using Speak, Duolingo Max, Practika AI, and Mac's AI assistant.
  • Participants completed identical dialogue tasks on each prototype so that group differences could be compared.
  • Based on a preliminary survey, participants were divided into high and low spoken-English proficiency groups.
  • The experiment was designed to closely simulate a real learning environment to ensure ecological validity.

05

Results

Clear differences emerged across AI interaction types. Real human-based interfaces yielded the highest immersion and satisfaction but also the greatest cognitive load. Animal and human character AIs supported stable engagement, while voice/text-based AIs offered a low-burden experience for beginners.

H1 Insight

Real-human interfaces increased immersion and satisfaction but also imposed higher cognitive load.

Average scores by interface type (5-point scale)

ImmersionCognitive loadSatisfaction
A Voice / Text · B Animal character · C Human character · D Real human
View as table
TypeImmersionCognitive loadSatisfaction
A · Voice / Text3.652.183.75
B · Animal character3.502.373.50
C · Human character3.432.683.05
D · Real human3.802.573.85

H2 Insight

Beginners preferred text/voice-based AI, while advanced learners preferred real human-based AI.

Scores by proficiency level (5-point scale)

BeginnerAdvanced

Immersion

Cognitive load

Satisfaction

A Voice / Text · B Animal character · C Human character · D Real human
View as table
TypeImmersion · BeginnerImmersion · AdvancedCognitive load · BeginnerCognitive load · AdvancedSatisfaction · BeginnerSatisfaction · Advanced
A · Voice / Text3.803.502.102.264.003.30
B · Animal character3.303.702.422.323.403.60
C · Human character3.003.862.802.563.003.10
D · Real human3.404.202.802.343.504.30
  1. 01ParticipantsOne hundred adult learners were split into high and low proficiency groups by a preliminary survey and took identical learning sessions across the four interfaces.
  2. 02Quantitative analysisA repeated-measures mixed-design ANOVA tested differences in immersion, satisfaction, and cognitive load across interface types and proficiency levels.
  3. 03Qualitative analysisIn-depth interviews surfaced the key experiential factors behind immersion and cognitive load.
  4. 04Hypothesis testingFive of the proposed hypotheses were supported and one was partially supported.
  5. 05Design implicationsConfirmed the need for interface strategies tailored to learner proficiency.

06

Proficiency-based UI

To improve learning outcomes, a 15-minute English placement test measures each user's proficiency before lessons begin. The AI then classifies learners by level and serves a UI optimized for it: low-burden text tasks such as fill-in-the-blank for beginners, and free conversation with a real-human tutor for advanced learners.

Placement test, beginner and advanced UI, and level result screens
From left: placement test → beginner and advanced lesson screens → level result.

07

Lessons Learned

Language learning applicability

The balance between immersion and cognitive load extends to other language-learning platforms beyond the apps used here.

AI tutoring enhancement

The design framework can expand into multimodal interfaces for classroom and personal tutoring.

Personalized learning

Adaptive interfaces can tailor learning paths based on learner proficiency.

08

Full Paper

The complete research paper includes methodology, statistical analysis, and design implications across all four AI interface conditions.

View Full Research Paper ↗