A sample KONSİL report
There is no form, keyword list or field selector. The discipline is inferred from the sentence and from the asker's publication record. This idea is education research; the asker is a radiologist.
Partially novel within the scope of this screen
The idea was split into four claims. Each claim was searched separately and given its own status with the closest paper. This is a model assessment justified with sources and quotations: the existence of each source and its quoted text are verified, the relation between claim and paper is judged by the model. It is a title-and-abstract pre-screen, not a systematic review; papers outside the screen may exist.
Structured peer feedback improves argumentative writing quality compared with instructor-only feedback.
This has been tested with a randomised design in large online university courses.
Measurement by blinded rubric scoring of argumentative writing.
A single randomised trial run across several universities simultaneously.
Why this assessment? The report's own reasoning
A defensible gap; the study answers a new question
Three of the four claims stood after the screen, so the gap status is defensible and the study frame is answering a new question. The gap as the report states it:
Multi-site, two-arm randomised trial with blinded scoring
| Research question | In large online university courses, does structured peer feedback with guidance prompts produce greater improvement in argumentative essay quality than instructor-only feedback, as judged by blinded rubric scoring? |
|---|---|
| Design | Multi-site, two-arm parallel randomised controlled trial; student-level randomisation stratified by university and course; pre/post argumentative essays scored blind against a common analytic rubric |
| Data source | Prospectively collected student essays and LMS feedback logs from participating large online courses, plus routine administrative covariates with consent |
| Primary outcome | What: post-intervention argumentative essay quality · Instrument: ⟦shared analytic argumentation rubric to be selected or adapted⟧, blinded double-scored · When: end of the feedback-revision cycle, same week in all sites · Unit: total rubric score · Meaningful difference: ⟦to be set⟧. Bracketed items are unknown and were not invented. |
| Analysis | Intention-to-treat mixed-effects ANCOVA on the rubric total, pre-test as covariate, random intercepts for university and course; site-by-arm interaction test; multiple imputation for missing post-tests |
| Open decisions | 5: rubric choice, randomisation unit, number of sites and sample, feedback-dose standardisation, adherence handling |
| Minimum team | 6 people |
The report lists 5 constraints, three required and two recommended: prospective registration and ethics approval at every site before enrolment; raters blinded to arm, site and pre/post status with essays anonymised and order-shuffled; a pre-registered analysis plan naming the primary outcome, model and minimally important difference before any scoring; a standardised, documented instructor-only comparator; and inter-rater reliability and protocol adherence reported per site.
Feasibility, in the report's words:
Four roles, twelve candidates
Roles are derived from the protocol. For each role the publication graph was searched for people who work on that topic; 37 institutions were scanned and matches are scored on published output. The twelve candidates come from Turkey, the Netherlands, Sweden, China, Greece and the United Kingdom.
Writing pedagogy / higher-education researcher (principal academic lead)
Owns the feedback intervention design, rubric choice and the educational validity of the trial.
Biostatistician / educational measurement statistician
Multi-site randomisation, clustering by course and university, and a pre-specified mixed-effects ANCOVA with attrition handling.
Blinded rubric rating coordinator (assessment specialist)
Rater training, anchor calibration and blinding integrity decide whether the primary outcome is trustworthy.
Learning-management-system / instructional technologist
Implements identical randomised feedback workflows and adherence logging inside each university's LMS without leaking allocation.
Each candidate comes with a contact route on record and a first-contact draft in a language the two sides share; if they share none, the chat translates in both directions. Only candidates derived from open academic records appear here.
Power analysis: the effect size came from the literature this time
Method: comparison of two independent means, α = 0.05, power = 0.80. The effect size was taken from the 2018 meta-analysis (d = 0.46) and its source is written next to it, so the number can be challenged: 76 per arm, 152 in total. The method choice is a language-model suggestion and requires supervisor approval.
Classification, document skeleton and gap list
- Committee: Clinical Research Ethics Committee (+ TİTCK authorisation may be required) — the asker's jurisdiction is Turkey
- Design: prospective, interventional, human participants → CONSORT
- Consent: written informed consent
- Vulnerable group flagged: students in a dependent relationship with instructors and graders
- 17-section application skeleton; 33 fields to fill and 26 verify notes marked one by one
The report says so itself: this is a pre-draft, not the application.
Four suggestions ranked by topic volume
| Journal | Quartile | Papers in topic | Access |
|---|---|---|---|
| Assessment & Evaluation in Higher Education | Q1 | 237 | Subscription |
| Frontiers in Education | Q2 | 204 | Open access (APC charged) |
| Language Testing in Asia | Q2 | 99 | Open access (APC charged) |
| International Journal of Assessment Tools in Education | Q3 | 26 | Open access |
Ranking follows the journal's publication volume in this topic in an open catalogue (OpenAlex), not citation prestige. A starting shortlist, not editorial advice; outside medicine the fit is weaker.
Output of this run
- One claim wounded with a citation before any data was collected; three left standing in this screen.
- A defensible gap stated in one paragraph, and a study framed as a new question.
- A nine-field protocol with a structured primary outcome, an analysis plan and five open decisions.
- Four roles and twelve reachable people, a literature-sourced sample size, and a 17-section ethics pre-draft.
Try it on your own idea
Write the idea in a sentence or two. The report checks it against the literature, derives the roles the study needs and finds reachable researchers. Closed beta, free; an ORCID iD is required.
Apply for the betaTurkish version: Örnek rapor, a different run in radiology education.