KONSİL

A sample KONSİL report

Real run, 7 September 2026. Every number and sentence on this page comes from that run; it is a frozen, dated example.
Published with the consent of the person who ran it. Candidate names, ORCID iDs and contact details are masked in this copy; the real report shows them with the source of each address. Source papers are open records and are given with their DOIs. Cards derived from KONSİL member profiles never appear in sample reports.
Input
"Whether structured peer feedback in large online university courses improves argumentative writing quality compared with instructor-only feedback, measured by blinded rubric scoring in a randomised design across several universities."

There is no form, keyword list or field selector. The discipline is inferred from the sentence and from the asker's publication record. This idea is education research; the asker is a radiologist.

1. Novelty assessment

Partially novel within the scope of this screen

Assessment: Partially novel within the scope of this screen 14 queries 83 source records 75 abstracts screened 3 full abstracts opened

The idea was split into four claims. Each claim was searched separately and given its own status with the closest paper. This is a model assessment justified with sources and quotations: the existence of each source and its quoted text are verified, the relation between claim and paper is judged by the model. It is a title-and-abstract pre-screen, not a systematic review; papers outside the screen may exist.

Close work exists

Structured peer feedback improves argumentative writing quality compared with instructor-only feedback.

Closest paper: "The impact of formative peer feedback on higher education students' academic writing: a Meta-Analysis" (2018), doi:10.1080/02602938.2018.1545896. Twenty-four quantitative studies; peer and teacher feedback produced similar gains (g = 0.46, confidence interval crossing zero). The idea assumed peer superiority.
Not found in this screen

This has been tested with a randomised design in large online university courses.

Not found in this screen

Measurement by blinded rubric scoring of argumentative writing.

Not found in this screen

A single randomised trial run across several universities simultaneously.

Why this assessment? The report's own reasoning

"The 2018 meta-analysis examining 24 quantitative studies found no significant difference between peer and instructor feedback (g = 0.46 with confidence interval crossing zero), directly wounding the core claim. However, multiple 2021–2026 papers have tested structured peer feedback with randomized designs and argumentative writing using rubric assessment in online or blended environments at single sites. No retrieved paper reports a randomized trial of structured peer feedback across multiple universities simultaneously with blinded rubric scoring as the primary design — this multi-site angle appears absent from the literature."
2. Gap status and study frame

A defensible gap; the study answers a new question

Three of the four claims stood after the screen, so the gap status is defensible and the study frame is answering a new question. The gap as the report states it:

"Whether structured peer feedback (with guidance prompts or scaffolding) produces superior argumentative writing gains compared to instructor-only feedback when implemented across multiple universities in large online courses, measured by blinded rubric assessment in a single randomized design. Prior work shows similar peer and instructor effects at single sites; replication across multiple institutions with this specific design is not documented in retrieved literature."
3. Proposed study

Multi-site, two-arm randomised trial with blinded scoring

Research questionIn large online university courses, does structured peer feedback with guidance prompts produce greater improvement in argumentative essay quality than instructor-only feedback, as judged by blinded rubric scoring?
DesignMulti-site, two-arm parallel randomised controlled trial; student-level randomisation stratified by university and course; pre/post argumentative essays scored blind against a common analytic rubric
Data sourceProspectively collected student essays and LMS feedback logs from participating large online courses, plus routine administrative covariates with consent
Primary outcomeWhat: post-intervention argumentative essay quality · Instrument: ⟦shared analytic argumentation rubric to be selected or adapted⟧, blinded double-scored · When: end of the feedback-revision cycle, same week in all sites · Unit: total rubric score · Meaningful difference: ⟦to be set⟧. Bracketed items are unknown and were not invented.
AnalysisIntention-to-treat mixed-effects ANCOVA on the rubric total, pre-test as covariate, random intercepts for university and course; site-by-arm interaction test; multiple imputation for missing post-tests
Open decisions5: rubric choice, randomisation unit, number of sites and sample, feedback-dose standardisation, adherence handling
Minimum team6 people

The report lists 5 constraints, three required and two recommended: prospective registration and ethics approval at every site before enrolment; raters blinded to arm, site and pre/post status with essays anonymised and order-shuffled; a pre-registered analysis plan naming the primary outcome, model and minimally important difference before any scoring; a standardised, documented instructor-only comparator; and inter-rater reliability and protocol adherence reported per site.

Feasibility, in the report's words:

"As written this is a funded 3-5 site education trial needing an education-research principal investigator and a rater team — far outside a radiologist's solo scope, so either co-lead it with an education faculty member or start with a single-course pilot to earn a seat at that table."
4. Team

Four roles, twelve candidates

Roles are derived from the protocol. For each role the publication graph was searched for people who work on that topic; 37 institutions were scanned and matches are scored on published output. The twelve candidates come from Turkey, the Netherlands, Sweden, China, Greece and the United Kingdom.

Role 1 · Domain

Writing pedagogy / higher-education researcher (principal academic lead)

Owns the feedback intervention design, rubric choice and the educational validity of the trial.

Top candidate: •••• •• Middle East Technical Universityh-index 17TurkeyFit 94/100
Match reason from the run: "Publishes directly on online peer feedback in higher education, including a 2023 synthesis of that literature and 2020 work on dialogic peer feedback design." Field match: strong · Reachability: high · Shared language: Turkish. Two more candidates, in the Netherlands and Sweden.
Role 2 · Method

Biostatistician / educational measurement statistician

Multi-site randomisation, clustering by course and university, and a pre-specified mixed-effects ANCOVA with attrition handling.

Top candidate: •••• •• Shenzhen Universityh-index 15ChinaFit 80/100
Shared language: English. Backups at Koç University (Turkey) and Umeå University (Sweden).
Role 3 · Measurement

Blinded rubric rating coordinator (assessment specialist)

Rater training, anchor calibration and blinding integrity decide whether the primary outcome is trustworthy.

Top candidate: •••• •• Middle East Technical Universityh-index 17TurkeyFit 90/100
Backups at University Medical Center Utrecht (Netherlands) and East China Normal University (China).
Role 4 · Execution

Learning-management-system / instructional technologist

Implements identical randomised feedback workflows and adherence logging inside each university's LMS without leaking allocation.

Top candidate: •••• •• Anadolu Universityh-index 16TurkeyFit 90/100
Backups at Hellenic Open University (Greece) and the University of Oxford (United Kingdom).

Each candidate comes with a contact route on record and a first-contact draft in a language the two sides share; if they share none, the chat translates in both directions. Only candidates derived from open academic records appear here.

5. Sample size

Power analysis: the effect size came from the literature this time

Method: comparison of two independent means, α = 0.05, power = 0.80. The effect size was taken from the 2018 meta-analysis (d = 0.46) and its source is written next to it, so the number can be challenged: 76 per arm, 152 in total. The method choice is a language-model suggestion and requires supervisor approval.

6. Ethics pre-draft

Classification, document skeleton and gap list

The report says so itself: this is a pre-draft, not the application.

7. Journals

Four suggestions ranked by topic volume

JournalQuartilePapers in topicAccess
Assessment & Evaluation in Higher EducationQ1237Subscription
Frontiers in EducationQ2204Open access (APC charged)
Language Testing in AsiaQ299Open access (APC charged)
International Journal of Assessment Tools in EducationQ326Open access

Ranking follows the journal's publication volume in this topic in an open catalogue (OpenAlex), not citation prestige. A starting shortlist, not editorial advice; outside medicine the fit is weaker.

Output of this run

Try it on your own idea

Write the idea in a sentence or two. The report checks it against the literature, derives the roles the study needs and finds reachable researchers. Closed beta, free; an ORCID iD is required.

Apply for the beta

Turkish version: Örnek rapor, a different run in radiology education.