Lightbulb Moments

TEA in Istanbul: Lightbulb moments from the delegates (Thursday Feb 16^th)

• The first thing that really struck me was the need for more academic stuff for my testing.
• Useful websites like www.lexile.com
• Having the chance of meeting people from different places with great experiences

Yeşim Ensari, Işık Schools

Things that I will try to use;

• Preparing headings for note-taking in listening tests
• Not to be obsessed with introductions and conclusions in an essay
• The need to write specifications before preparing a test

Mazal Dozbahar, Işık Schools

• Tests should all be context specific
• Thesis statements were probably  invented by bad writers!

ANON

I have benefitted from the workshops in Tea in Istanbul in the ways below:

• How to write better quality items
• How to fit them on the CEFR
• How to better plan before starting to design a new measurement tool
• How to fine tune levels of difficulty in reading texts

Huriye Sümer, Yaşar University

• www.lexile.com
• the importance of validity
• how to choose an appropriate text for a test
• writing different types of questions

Ayşem Mert, Işık Schools

• In matching items, for instance if you are asking 5 items, make sure to ask 5 extra options, otherwise it will be a sort of double jeopardy!
• In SAF, limiting the number of words in the response prevent copying and pasting
• EAQUALS – the Core Inventory for language

Aylin Geron, Işık Schools

• Vocab Profile for analysis of lexis in a text
• Three option MCQs
• Test specifications- very important
• 3-5 words for short answer questions
• Minister for exam poem

Gülce Bayrakçı, TED University

• Specification writing is much easier with the CEFR
• Deeper look at global proficiency tests is important when selecting one

ANON

“Every test is a series of compromises, no test is perfect.” BOS

Ahu Dereli, Istanbul Ticaret University

• As a group we understand that the concept of specifications is the key and has a great importance when preparing exams to ensure standards
• Try to avoid using so many multiple questions in our exams. Open ended questions and SAF are much more reliable in assessing students  success
• Preparing a reading framework is crucial

Muge Tokman, Istanbul Ticaret University

• In our institution with limited resources and skills, the outsourcing of appropriate testing to recognized and established test providers while not perfect is a viable short term option

ANON

• The importance of making concrete test specifications
• Put “can do” statements on the students’ websites
• Make student friendly rubrics/criteria
• Don’t measure grammar in speaking assessment interaction tests

Peter Campbell, Beykent University

• Analysing the reading texts you intend to use, using computer programmes is much more reliable than depending on your instinct.
• Using MCQ type questions should be reconsidered!
• Everything that is related to testing is about your context. Analyse your context carefully and then you know really what you are testing.

Yalçın Ünal, Hacettepe University

Some terms I have encountered

• Validity
• Specifications
• Confluency
• Expeditious reading
• Europass (European Extended CV developed by Council of Europe)

Sevgi Şiranli, Işık Schools

• 95-98% of words need to be understood for text comprehension

ANON

• Test specifications need to be developed asap
• Different sets of criteria for production and interaction tasks in speaking tests
• Coh-metrix and Lexile websites.
• Setting standards for text difficulty in the tests according to the levels in our context

Selen Şirin, Özyeğin University

• SAF (short answer format) should not exceed 6 words.
• Try to avoid MCQ(Multiple Choice Questions) format
• Test specifications are an indispensible part of the test preparation
• International tests are not always reliable

ANON

• There are many things to consider especially when assessing the productive skills
• Descriptor play a crucial role in testing
• Listening is the most difficult skill to test

ANON

• Different scales for different tasks
• Testing is a team effort
• Reliability equals consistency

ANON

• More tasks ensure a higher reliability than having more raters.
• Most of the time, raters listen to errors and disregard other essential components of the criteria like range
• Detailed reading questions-using exact words from the question turns the task into a matching activity
• Don’t have more than 3 options in MCQ format

ANON

• EAQUALS has some faboulous resources available for everyone online

ANON

• British National Corpus (BNC20) @lextutor.ca
• The idea of certainty elements to T/F questions
• Steps to prevent cheating internationally on public exams
• You can’t really compare scores on different tests unless you do it item by item
• It is impossible to get a completely accurately test.

ANON

• Testing is validity
• Testing is making decisions,
• For real interaction in a speaking exam, the examiner needs to be prepared to let the examinee to take control
• There is no bets way of testing. You as the institution know what is best in your context, your resources and your purpose

Emine, Ebru, İzlem from Bilgi University

• The validity frameworks for different skills
• Text complexity and the Core Inventory- British Council and EAQUALS

Mohammed Ghazwan, Bahrain Polytechnic

• We do so much need to start writing specifications to develop proper classroom assessment

ANON

• Some practical ways of assessing text difficulty (lexical, syntactic) and setting a range for use in specifications
• A potential place for L1 in areas such as rubric /summarising

James Brannan, Doğuş University

• The workshop helped to clarify how to use the CEFR book
• The workshop provided lots of information that allowed us to find answers to our own questions related to our institutional needs.
• The workshop participants were divers and this was good as it allowed exchange of ideas

Pembe Garip, Eastern Mediterranean University

• We need to use more university texts instead of texts taken from newspaper articles, etc.
• A test itself isn’t valid; the inferences we draw need to be valid
• Using coh-metrix to analyse text is a great idea

Özlem Atalay, METU

• Analyzing texts that students use in undergraduate programmes for lexical difficulty, choosing and analyzing text  for tests in the prep programme accordingly
• Using the Core Inventory for General English for syllabus, revision and test writing

Dilek Eryılmaz Salki, Özyeğin University

• Creating a test is along process and needs to be planned effectively taking into account the teaching context, institutional needs and realities as well as the students’ characteristics.
• Test specifications are crucial.

Çiğdem Tuncergil, Eastern Mediterranean University

• Specifications need to be dealt with much more emphasis
• Despite the differences in terms of student numbers, physical conditions, etc. we all think in a similar way re assessment
• We should reconsider asking vocabulary and reference questions while assessing reading
• Double checking in high stake exams is necessary and we should find a way of doing it

ANON

• Keep it practical above all else
• Institutional proficiency exams- interesting concept (as opposed to using IELTS or TOEFL as an external measure

ANON