Speaking Practice Transfer: A Replicable Evaluation Protocol

A speaking intervention shows transfer when learners improve on a new but construct-relevant communication task, not only on the prompt, script, or feedback examples they practised. A credible evaluation predefines the target population and claim, separates exposure from outcome, uses comparable baseline and follow-up tasks, adds an untrained transfer task and delayed measure, and publishes missing data, uncertainty, harms, and null results.

Scope and method

This protocol was assembled from sources searched and accessed on 29 July 2026. It uses official CEFR descriptors to define functional task claims, peer-reviewed pronunciation and fluency measurement work to choose outcomes, and corrective-feedback research to distinguish immediate task effects from later learning.

The source scope included adult and adolescent second-language speaking, communicative task design, fluency measurement, pronunciation measurement, and framework descriptors. Exclusions were vocabulary-only outcomes presented as speaking transfer, product satisfaction without performance data, unversioned automated scores, and before/after comparisons with no stable task definition.

This is a proposed protocol, not a registered trial, consensus standard, or report of LinguaLive outcomes. It must be adapted and prospectively registered before it is used as a study plan.

Step 1: write the claim before choosing metrics

Use a population–intervention–comparison–outcome–time statement. For example:

Among adult B1-level Polish learners of English preparing for workplace meetings, does six weeks of scenario rehearsal plus delayed feedback, compared with the existing self-study programme, improve completion of an untrained scheduling-negotiation task one week later?

The statement does not promise broad fluency. It identifies who, what changed, what the comparator receives, which speaking function matters, and when the result is tested.

The Council of Europe presents CEFR levels through many illustrative can-do descriptor scales rather than one countdown metric (official descriptor portal). The CEFR Companion Volume adds scales for production, interaction, mediation and related strategies. Descriptors help define tasks; self-reported “can do” responses are not a substitute for observed performance.

Step 2: define trained, near-transfer, and far-transfer tasks

Task layer	Example	Purpose
Trained	Rehearse rescheduling Tuesday's project review with known details	Confirm the intervention was delivered and the target was attempted
Near transfer	Reschedule a medical appointment with new constraints and a new partner	Test the same function with changed language and setting
Farther transfer	Negotiate responsibilities after an unexpected team delay	Explore broader use without claiming equivalence

The near-transfer task should preserve the communication function while changing surface details, prompt order, and exact vocabulary. A far-transfer task is more ambitious and should be reported separately. Failure to improve far transfer does not erase a valid trained-task gain; it narrows the claim.

The role-play scenario design guide explains how to change constraints without producing a keyword-swapped script.

Step 3: lock the measurement plan

Use an outcome family that matches the claim:

• task completion: required information exchanged, option compared, and next action confirmed;
• intelligibility or comprehensibility: independent listeners using an anchored rubric;
• selected language target: accuracy or appropriacy defined before analysis;
• temporal delivery: speed, breakdown, and repair measures where relevant;
• interaction: clarification, response relevance, turn management, and repair;
• adverse outcomes: false correction, withdrawal, accessibility failure, or inappropriate content.

Saito's pronunciation measurement framework demonstrates that construct, scoring method, and elicitation task change what an outcome represents (measurement meta-analysis and accepted manuscript). Current oral- fluency research likewise warns against collapsing speed, pauses, repair, and listener judgement into one number (assessment timeline).

Use the oral-fluency methods brief for annotation choices. If automated features contribute, apply the fairness validation map before the score affects a decision.

Step 4: preserve comparability without teaching the test

Create parallel task forms and pilot them outside the main sample. Keep instructions, planning time, response window, partner role, audio setup, and rubric stable. Counterbalance task forms where feasible. Do not show raters whether a recording is baseline or follow-up.

Record intervention exposure separately:

• eligible days and sessions offered;
• sessions and target attempts completed;
• verified active speaking time;
• feedback events opened and acted upon;
• partner or model version;
• interruptions, outages, and missing recordings.

Scheduled minutes and page views are not spoken practice. Exposure explains whether the intervention occurred; it is not itself proof of learning.

Step 5: plan allocation and sample size

Choose random assignment when feasible. If programmes or classes are assigned as groups, account for clustering. If randomisation is impossible, document the selection process, baseline differences, concurrent instruction, and why the comparison is credible.

Calculate sample size from a primary outcome, minimally important difference, expected variation, design, attrition allowance, and analysis plan. Do not reverse-engineer a target from the number of available users. Small pilots can test feasibility and measurement reliability, but should not be promoted as definitive efficacy studies.

Step 6: register analysis and stopping rules

Before looking at outcomes, specify:

1. one primary outcome and timepoint;
2. secondary and exploratory outcomes;
3. inclusion, withdrawal, and missing-data rules;
4. treatment of unusable audio and failed automated scores;
5. model, prompt, rubric, and software versions;
6. subgroup analyses and the minimum sample needed to report them;
7. correction for multiple comparisons where applicable;
8. adverse-event review and pause criteria;
9. effect estimates and confidence intervals to report;
10. a publication plan that includes null and adverse findings.

A long metric menu does not compensate for an ambiguous primary claim.

Step 7: interpret transfer conservatively

If trained and near-transfer tasks improve but the delayed outcome does not, the finding is a short-term task effect. If temporal fluency changes but listeners do not understand more and task completion is flat, report the timing change. If one subgroup has a high automated-score failure rate, do not average the problem away.

Corrective feedback can affect immediate use differently from later learning. The feedback-timing evidence brief explains why uptake inside a session is not a delayed transfer measure.

Minimal reporting table

Item	Required disclosure
Population	Recruitment, eligibility, language background, level evidence
Intervention	Version, tasks, feedback, dose offered and received
Comparator	What participants actually did during the same period
Outcomes	Construct, task form, rubric, rater or model, reliability
Transfer	Distance from trained task and rationale
Time	Immediate and delayed measurement points
Analysis	Assignment unit, exclusions, missing data, uncertainty
Equity	Accessibility, subgroup coverage, abstention and error patterns
Interests	Funding, product ownership, analyst independence
Availability	Protocol, materials, code, de-identified data constraints

The LinguaLive speaking tools can supply rehearsal opportunities for a future independently reviewed pilot. Their availability does not establish the outcomes in this protocol, and LinguaLive should not describe a practice session as an official assessment.

Limitations

No single protocol fits every language, learner group, or institution. CEFR descriptors are illustrative references, not a universal assessment battery. Interaction tasks add partner effects that monologues avoid but real communication requires. Blinding participants to a speaking intervention is rarely possible. Long-term and far-transfer measures increase attrition and cost.

This protocol does not provide ethics approval, legal compliance, a sample-size calculation, or a validated rubric. It does not establish that LinguaLive or any other product improves speaking.

Editorial disclosure

LinguaLive could benefit commercially from favourable trial results. For that reason, a consequential study should use independent protocol review, analysis, and outcome rating where possible, with conflicts stated in the registration and report. Vlad Podoliako is named as LinguaLive's founder and does not claim research-methods or psychometric credentials. Null, mixed, and harmful results must remain publishable.