Warnings of problems hitting Sats

The setting and marking of the tests is a huge exercise

Schools and markers say there are problems with this year's national curriculum tests or "Sats".

Some schools could not access the registration system they need to complete so that markers know which pupils' work they should have.

Markers' complaints include a lack of organisation and communication by the company running England's school tests this year for the first time, ETS.

The National Assessment Agency (NAA) says it is confident all will go well.

ETS (Educational Testing Service) is a US-based non-profit organisation with extensive test administration experience in some 180 countries.

It was awarded a five-year £156m contract by the NAA a year ago. This covers the tests that all of England's 11 and 14-year-olds are taking and the Year 7 progress tests.

Contracts

This week, schools found they could not complete an online register which is vital so that markers know which pupils' work they should receive.

A statement on the NAA website said it was aware some schools had been experiencing problems and was investigating.

A very senior marker told me he thinks the Sats marking for English is close to total collapse First-time marker

But other problems have been identified by markers and would-be markers for the past few months.

Posting in forums on the website of the Times Educational Supplement, they highlighted what one called "major disorganisation problems".

When contracts for the work were sent out by ETS, some markers were told they had been chosen on the basis of their previous year's performance - even though this was the first time they had done it.

Others were sent a contract for the wrong subject - or more than one contract.

Sometimes efforts to contact ETS by phone or e-mail went unanswered.

In some cases people's references were taken up after contracts had been sent out.

One marker received a request for a reference about her - sent to herself at her home address.

There was confusion over training venues, complaints about the training itself and the late dates at which everything was being done.

Problems seem to have been compounded by a new procedure this year intended to tighten up the selection of suitable markers.

They had to pass an online "standardisation" assessment of their marking consistency. They were given two attempts to pass this or face being struck off as a marker.

But those who attempted it last weekend were initially all told they had failed - even if they had not.

'Resolved'

The NAA said this had affected only some 200 out of some 5,000 markers and they would all be contacted to explain the situation, with those who had indeed failed at the first attempt being given a second chance as planned.

Markers and schools can be assured we are giving all areas of concern our immediate attention to ensure a successful 2008 cycle Andy Latham, ETS Europe

A spokesman said: "NAA is confident that the external marking of the key stage statutory tests will be accurate and results will be returned to schools according to the published timetable.

"Where there have been any small localised administration arrangements which have caused a delay in marker training these have been resolved and the individual markers contacted."

But one first-time marker told the BBC News website: "ETS had promised senior markers they would clarify every marker's pass/fail status by email (because many people were failed in error). This has not happened."

He added: "A very senior marker told me he thinks the Sats marking for English is close to total collapse.

"As far as I am aware, even people who have passed have not even been allocated scripts to mark."

'Resolution shortly'

ETS Europe said in a statement it was aware "that some operational aspects of the national curriculum testing process have not gone as smoothly as planned".

It apologised for the inconveniences experienced by some markers but said steps had been taken to correct all issues "and resolution is expected shortly".

Andy Latham, ETS Europe vice-president for NAA Projects, said the capacity of its helpline for markers had been increased.

"Markers and schools can be assured we are giving all areas of concern our immediate attention to ensure a successful 2008 cycle."

In a statement, the new exams regulator, Ofqual, told the BBC there were some significant changes to the "delivery" of the tests this year.

"We are aware of some problems and have been monitoring these issues to ensure that the pupil experience is as it should be and that all pupils get the results that their work deserves."

Liberal Democrat schools spokesman David Laws said there was a very real worry whether this company had made the proper preparations for the exam papers to be marked effectively.

"The government should be intervening to make sure that these tests will be properly marked, but instead ministers seem to be washing their hands of any responsibility for the mess."

I was told we had to attend a training session on 10th May in order to be 'allowed' to mark SATs, but they failed to tell me where to attend. A week later, I still haven't heard anything about the extra session they told me they were laying on for those who missed out. I received two references for myself sent to me at my home address, calling me the Head of English and Headteacher. When I phoned ETS I was told not to worry and got sent the contract anyway, even though I knew they hadn't contracted my referees...
Eve Blakeman

I am a science KS3 marker and would like to say for me everything has gone fine so far. My Team Leader has been very supporting and my first batch of scripts were delivered today. The strict standardisation guidelines have meant that I have come to know the mark scheme very well which I am sure will help me know end. I would also like to point out that I also received a letter telling me that I was selected on last year's performance despite me not marking last year. I did what any sensible proffesional should do, assumed it was a clerical error and got on with my life! It was no big deal.

As for using the excuse of teacher's busy schedules, come on we are getting paid to do this, no one is forcing you. If any of you thought it was not worth the money you wouldn't be doing it. Yes the system isn't perfect, no system is. Let's just get on with it and stop living up to teacher's reputations as moaners!
Robert Jones

My experience of communication with ETS has been very poor. Inititially I received two contracts - both of which told me I had been chosen on the basis of my previous year's performance. I have never marked SATs tests before. This should have warned me of the problems to follow. Despite numerous emails and attempted phone calls to ETS, I still have no information regarding venue for TOMORROW's training session. I am unable to speak to anyone on the phone as I am continuously put in a queue.

I was told my team leader would contact me directly to introduce him / herself, but this has not yet happened. I am close to despair with ETS, and really wonder whether some children's test results are going to be adversely affected by this total shambles.
Nicola, Manchester I have marked KS3 maths SATs for more than ten years, but this year the organisation seems to have been a complete mess. The only correspondance i have had was an e mail saying i was in Wolverhamplton, a letter , which arrived the day before the training day expecting me to drop everything and organise a 600 mile round trip down to Wolverhampton the next day. i knew that there was a training meeting 15 miles away but it just seems ridiculous that they seem to be more willing to pay for overnight accommodation and a huge amount on a last minute rail fare rather than organise more places to train locally. I know of at least 6 other markers who have declined to mark because of the distance they will need to travel or because of the poor organisation of ETS.
S Williamson

Totally appalling service. I am lost for words. I have just withdrawn my application due to the complete incompetence of ETS. Emails and phone calls not returned, phone calls demanding why I haven't returned documents when I haven't even received them... and finally an offer for KS2 science training session on Monday, 150 miles from my home with 2 days notice- do they not think teachers go to work???
Joanna

As an ex headteacher I applied and was accepted as a KS2 Science marker,received some initial information (in which ETS identified me as a previous marker, though I was not),I signed & returned my contract to ETS Europe. Then nothing, SATs approached. I emailed ETS. ETS emailed to me a timetable of events but I could not access on-line pre training day admin. tasks etc.I had to request username and password in order to log on to site. SATs began and I had NEITHER been contacted by a team leader NOR had I received any notification whatsoever of the training day. I received a 'phone call from ETS on Moday evening (of test week) the outcome being that the ETS operative did not have me on his list and I would be called back. Guess what? Nothing. Picked up my emails yesterday morning (Thursday of test week) and found message from ETS informing me of training day details for Saturday - 2 days hence. I have declined to attend the training and have withdrawn my request to be a marker as I do not wish to be associated with such an appalling organisation.
Richad Ash,

As Head of a primary school I am deeply concerned that so little thought and care has been taken with the SATs service to schools. How a Government can put such an obscene emphasis on SATs and the ensuing results whilst at the same time take so little care on the contractors who supply them.
Andrew Fielder

As an LA [local authority] representative on assessment I sincerely feel the system this year has already broken down and caused a huge amount of panic, stress and feelings of inadequacy. The pressure placed on pupils is high and can often be minimised by teachers and other school staff - if the teachers and school staff are feeling so anxious, the pressure on children and the stress and worry they feel heightens. I think we all need to remember that if it wasn't for the children we wouldn't be here - GIVE THEM A BREAK! ETS, in my opinion have underestimated the profile of the SATS and many headteachers have already commented 'what was wrong with the good old paper system?'
Jonathan Whitfield

All kinds of problems - asked to fill in a second contract, told no contract, then wrong subject contract, asked to return copy of contract sent - no contract arrived. 6 emails and umpteen voice mails received no answers. In the end I contacted Andy Latham yesterday after being told i was not on the database for KS2 training day on Saturday. He replied almost immediatley giving details of a training venue in Leeds ( I live in West Wales). Sent email turning down the venue and half an hour later had second email from Andy offering me a Bristol venue. The helpline was hopeless - I was lucky knowing who to contact first in NAA ( who directed me to Andy Latham, ETS) and then having swift response from ETS, together with apology for the unnacceptable delays I had experienced. However, my daughter has just been told she has failed her online standardisation when she was not even able to get onto the website to attempt the standardisation test. No KS scripts have arrived.
Anne Griffiths

Absolutely unbelievable ETS "helpline" - as a marker - I was given a training venue 600 miles roundtrip (very late notice too) - ETS unable to change my venue - did not reply to any emails (after 7 days), totally unhelp on "helpline".

School also has not yet received the stationary necessary to send back the tests (4 days worth already completed) UPS did not arrive today for "pick up". Suggestion - CDC (central distribution centre) is still overcrowded with KS3 SATs - so maybe KS2 SATs will never be collected this year.
Helen Sheridan

I have marked exam papers for the last four years - Sats KS3 Reading for the last two. This year I found out about the training meeting less than 48 hours beforehand. I went & had no allocated team leader - there were many who had not turned up. I went home & tried the online standardisation - had problems when I went out of the script clusters & then tried to return - marks disappeared... called the helpline which is an 0870 number which I think is disgraceful. After a lengthy wait, the person at the call centre took my number and hung up. (Apparently this has happened to others). After far too much time, I completed the standardisation only to be told I'd failed - I then tried to contact the team leader I had been with on the training day. He contacted me the next day & said that the system was failing people incorrectly and that I wasn't in his team. He said I should wait for a new team leader to contact me. AT this point, after reading the experiences of others on the TES website, I decided not to continue as a marker. It is impossible to contact ETS & given that there has been such poor organisation, it is unlikely that I might be paid correctly or on time. Neither do I think that the support will be forthcoming if things go wrong with the technology during the marking process. 24 hours after emailing ETS to inform them that I would withdraw (I was an A grade marker last year), my new team leader contacted me.

My main concern is that the marks may well be compromised. I do not agree with the Sats process at all as I feel that it alienates pupils and doesn't benefit them educationally. However, this year I am extremely concerned - I have worked hard with my Year Nine class all year - my school, my department and the core subject teachers are judged on these results. Pupils' forecast GCSE predictions are based on these results - what I have seen so far seems highly unprofessional. The main benefit of taking part in the training and having experience as a marker is that I intend to rigorously check our English papers when and if they come back.
Mandy Thirsk

In short: problems with receiving contract after having been accepted (had to phone 4 times to get it sorted, late notification of training venue, now having passed standardisation on monday still waiting for scripts to be dispatched. under old system we would have received scripts 2/3 days after exam - now 2 weeks behind with new mark capture to do which will take more time, and we are expected to complete marking by 23 june as normal. ets help desk are not helpful because they don't have any information.
K Adamson

Having signed a contract with ETS to mark English KS3 Reading, I spent a whole day on the phone to them and to NAA trying to find out the venue for training. No-one could tell me this,therefore I have not been trained and consider that I am unable to mark.
Susan Breedon

I have been a Science marker since 2000. I am also a Primary Head Teacher who is now VERY worried about the effect this mis-management will have on the children who have had to endure yet another week of unnecessary political testing that feeds the league table farce. For the last two years I have been awarded an 'A' grade for the standard of my marking. I have this year resigned as a marker as I had left many messages and e-mails expressing my concern that I still did not know where I am supposed to be training THIS SATURDAY (17th May). No-one has replied by e-mail yet. I could not access the online training module that I was supposed to do before the training. I have decided that the stress and frustration that this has caused me has gone on far enough. I have never felt like this in any of the previous years.
Keith Wright

I was initially informed that the training meeting for team leaders would be in Birmingham. On the Tuesday before the Saturday meeting, a message was left for me saying that the venue had been changed to London. On the Wednesday I tried to confirm the message with ETS with no success, so I booked accommodation and train tickets. Three hours before I left for the station (on the Friday) I received a telephone call to say that I was to go to the Birmingham venue. I declined to change arrangements at such short notice and went to the London venue. The second team leaders' meeting followed a fortnight later, also in London. We had about 12 hours notice of a venue change (within London) for that meeting, which I was lucky to pick up on my mobile messaging service. The additional cost of sending me to London, rather than Birmingham - train fares, hotel bills etc - must be about £500.

If the NC tests work this year, it will be because the proper professionals - the teachers who mark them - make it work.
Stephen Davies

I have been contracted to mark KS3 English Writing. I was only told the venue of my training meeting a few days before it happened. Some people had to travel hundreds of miles at short notice. I did my standardisation on Sunday and was told that I had failed. I tried to withdraw from marking, however my team leader literally begged me to stay because they have not got enough markers. I was told on Tuesday evening by my team leader that I had in fact passed and should receive confirmation of this by ETS with an apology. This has not happened. I have not marked any scripts because none have been sent to me, although the ETS website shows that I have schools allocated. I am a team leader for GCSE at another exam board. I also mark A level. Both of this examinations need to be marked from next week. If ETS do not send me scripts soon, I will not be able to mark them because of my commitments to other marking. I believe that I am not the only marker who will not be able to marks SATS due to other examination commitments.
JC

I am a first time SATs marker, having opted to do KS3 English Reading rather than GCSE as I usually do. I received a phone call in mid-April, asking me why I hadn't returned my contract. I pointed out that I hadn't yet received a contract. They apologised and guaranteed that I should receive it by the end of the week and to call if I hadn't. I didn't receive it and so I rang and enquired. They replied that they would definitely send it out in the next week and, again, to call if I hadn't received it. Again, no contract arrived. I called again. They said they would send me one.

Two days later, they rang me. On a Sunday evening at 7pm. They left a message asking me to call back but no number to call them on. The helpline number was closed until Monday morning. I rang on the Monday morning and they asked why I'd not sent it back. I explained that I'd not yet received it. They agreed to email me a contract. 4 days later, it arrived. I printed it out, signed it and sent it back.

10 days later, the day before the exam, I finally heard from ETS. Was it to arrange training? No - to ask where my references were. When I said that they had been sent back weeks before, the man on the phone said "they're probably just not logged properly on our admin system, it happens a lot, don't worry".

The next thing that I heard was the day after the exam, when they rang to ask me to attend training. The training day was scheduled for 24hrs time and the journey would be a 600mile round trip. I refused attendance at such short notice for such a distant venue. I was told that I would be contacted to rearrange a venue. That was over a week ago - I have heard nothing, nor has my team leader. My emails go unresponded to, my phone calls are not answered... I have no idea whether I am expected to mark or not.
Jennifer Bull

I am an experienced KS3 maths marker and have applied and been accepted by ETS Europe as a marker this year. My training day was 10th May and I was not informed, like many other markers of the venue. I wish to vent my frustration at all the time and energy I have spent trying to contact ETS Europe (8 emails - and 'hundreds' of attempts at contacting the helpline - no response at all) to ascertain my training venue. I was intending to mark KS3 maths SATs this year but am now thankful as the true incompetence of ETS Europe come to light, that I am not involved in the whole fiasco. I did not find out the venue of my training (and so did not attend), but unlike others who have no team leader, I do have a team leader!!!!! I have recieved an email and phonecall from a very nice team leader who has been informed that I am part of his team! Amazing! Apparently my training was meant to have been in Sheffield where 10 team leaders turned up expecting around 100 markers. Allegedly only about 16 markers arrived (some of these were for KS2 papers)! ...
Karen Symons

Today we were due for the first collection of KS2 SATs by UPS. It seemed to go well as they were collected in the morning, but then this afternoon another UPS driver turned up to collect them.How many other schools have had duplicate collections?
C Hope

When I was doing the short writing paper my class got extra instruction sheets, and half the amount of answer sheets we should have. As a result, we had to delay while our teacher phoned up the NAA to get permission to photocopy the answer sheets that we did have.
Leila

I am a marker and have been for 12 years. This year I have failed, no passed, no failed, no passed the standardisation test! Now, a full week later, I still have no scripts to mark for KS3 English and find that many markers are in a similar poistion. A farce!
Melanie Cookson,

Marking KS3 Maths - passed standardisation online last Sunday. Still no papers and no advise on when they will be delivered (it's now thursday pm). This time last year I would have by now recieved the scripts (direct from the schools allocated to me) checked I'd received them all and be well on my way to marking them. I've lost a whole week - are they going to add a week onto my completion date? - I don't think so.
Philippa Peacock

Please do not use my name if comments published. I have pulled out of marking this year (after 4 very successful years) due mainly to ETS wanting me to attend training venue too far away when nearer ones were available. They refused to change the venue so I pulled out. THey also failed to send information by dates specified in contract and letters.
[name and phone number supplied]

Problems only affecting 200 markers - what a load of rubbish! You only need to look at the number of posts on the primary, english, maths & science forums on the tes website. I myself, have received 2 science contracts and 1 english contract, been assured I was marking ks2 english, and then today, 15 may, have been sent details for a ks2 science training day & also old I have a contract to mark science.
Claire Houghton

This is getting ridiculous, I passed standardisation for Maths Key Stage 3 4 days ago and as yet have received no scripts to mark. I have telephoned and emailed ETS without any success.

In the past I have always received my scripts by the Monday after the actual exams. If as now seems likely I will not receive any script Friday I will have lost a weeks marking time. With the standard number of scripts increased from 300 to 450 there is considerable risk that the end dates will not be achieved.
Ed Newns

I am due to mark KS2 Science however I am still to be assigned a training venue for the training on Saturday! ... If it wasn't so important it would be funny. Children have worked very hard to complete these SATs, they deserve to have they marked professionally by markers who have been trained.
Anna Wilson

Though awarded a contract to mark in January I was informed of the training date and venue 4 days before the event. I was unable to access various pieces of information required and as such have pulled out Last year I had scripts in hand within 24 hours of the examination, excellent practice material and a well planned induction. Total rubbish this year
Paul Mercer

I have been a KS3 marker for the past three years. In October, I re-applied to ETS on their website. I had no response from them for nearly 4 months, when I received a contract through the post in February. I immediately returned it to them, however received no response in return. I called ETS (with no response) daily for a week prior to the standardisation meeting. I eventually received a letter confirming the meeting time and date 3 days prior to the meeting. By this point, I had made other plans, and couldn't face the disorganisation of this company, so decided to resign, as had another A grade marker I work with who had experienced exactly the same.

I telephoned the company to resign and managed to speak to someone who informed me that the resignation had been processed. I subsequently received a telephone call and email from ETS on the day before the meeting asking me to confirm my attendance at the meeting.

ETS are a shambles. I am so glad I chose to resign, but am extremely concerned for the students who have sat their exams this summer, and fear that they will not receive their results in good time, and due to the changes in the marking process. I am also certain that the quality control has been negatively affected by the changes in administration method. I will definitely be checking the marks of the students I teach very carefully to ensure that the correct levels are awarded.
Deborah Stamp

I've marked KS3 SATs with Edexcel for years and never encountered anything like the disaster of the past week. ETS have been uncommunicative at every turn and left me feeling incredibly angry To be told you have failed a task that it central to your role as a professional is incredibly dispiriting, and the continued lack of communication from ETS has only compounded the issue.
Janet

I am a marker for Key Stage 3 Mathematics and the whole experience is a sham. Many people have given up and resigned and I am close to that myself. It takes up to 10 minutes to get through to the helpdesk and then you get told that a supervisor will be made aware of the problem you have. They never contact you again or reply to any emails. You phone again and just go round in circles. By this time last year I had made a significant start to my marking, at present I don't even have the scripts and there is no sign of them arriving soon.

They are also insisting that they deliver them to my home address when I work full time and never get home before the last delivery time!! They are refusing to send them to my school because "it is not safe". Yet they would be quite happy for me to leave them with a neighbour that I hardly know. My school is currently keeping "safe" several thousand exam papers which our GCSE, AS and A Level students are about to sit. For all I know my neighbour might set fire to them!!
Jo

Our school have had real problems with the online registration form to the extent that our head teacher was still trying to get on 10 o'clock last night.
Kath

You know what, this is exactly the same problem as the one with the KS3 ICT Online Tests, the ones that got binned a year ago after tens of millions were spent on it and thousands and thousands of teachers' time was used in 'training' and other pointless activities. The companies that get these projects are not properly, independently and objectively monitored for quality. What has happened with ETS, which is what happened with RM and the online tests just refered to, is that they are monitored by organisations that must keep say that everything is ok publically (whilst privately praying it is). The NAA are doing it here. Before with the tests, it was QCA.

After the fiasco of those KS3 ICT online tests, what *didn't* happen was an investigation into why the body overseeing that project was so positive everything was great, right up until the moment the plug was pulled. So no lessons have been learnt. We schools will have to put up with stupid idea after stupid idea, wasting millions of pounds of public money, and lots of time again and again and again and again. Until someone puts in a procedure of proper, independent quality control.
David Cooper, Head of ICT, Rugby High School

"Sometimes efforts to contact ETS by phone or e-mail went unanswered."

Sometimes! Emails direct to ETS remain routinely unanswered. The 'Help desk' operators know nothing. repeated email and telephone requests to verify training venue were a)ignored b)no-one knew but an email confirming promised repeatedly. None materialised.

On the web site I am an 'approved' marker, but clearly can't mark as I wasn't able to access training. I have lost a significant amount of money - potentially. ...
Susan

I have marked English reading for 2 years prior to this. I was offered a contract to mark, which I returned to ETS on two seperate occasions. 2 weeks before the training event, I hadn't received any correspondence from ETS regarding the imminent training event. I phoned and emailed them 3 or 4 times a day for 2 weeks to try and sort out the situation. Eventually, I was told that I was in a queue of around 20,000 problems, and would be dealt with as a priority. Ultimately, I was told on the evening before that I couldn't attend the training event because ETS couldn't tell me where I was supposed to be going. They offered to run a training day during the school week, but I refused this and withdrew from marking. The level of imcomptence and utter idiocy has been staggering. Glad I'm out of it - although I'm concerned that the results of the marking process will be lots of dodgy marks.
Alun Owen

One of my collegues is marking KS3 maths SAT's. Compared to last year he has been allocated 50% more as standard, the deadline is shorter and the papers are late. In previous years he would have had them by last weekend but has not yet received them, ETS have said that they could arrive "whenever"! The results will be late!
Rob Johnson

This is my 5th year of marking English Reading KS3. Every year I say I'm not going to do it again, but, inevitably, the money is needed to see me through the summer. From Lancashire, I was actually given a markers' meeting very close to home - in Manchester; two years ago, I was sent all the way down to Portsmouth, only to find, when I got there, the whole of my team had come from Lancashire like me! So...problems like we're having now from ETS are not 'new' ones! They are certainly being very strigent with the standardisation: I, like so many others, was told I'd failed...only to discover from my Team Leader that I have now passed (fishy!) How well the benchmarking will go after every 100 scripts marked, is anyone's guess...as is whether we'll be paid on time! I know hundreds of markers have decided enough is enough, however, this year, and have walked away, so I very much doubt the scripts will be marked on time.
Claire Banfield, Bury Grammar School, Girls

I am marking the KS3 English Reading scripts and can testify to similar experiences to those detailed above. Furthermore, I initially failed standardisation (and dispute the 200 figure - I personally am aware of about 20 people who failed.) Even though I have apparently failed, I have now been allocated scripts to mark - with no indication from ETS that I have actually passed. Delivery details are not available for some unknown reason. Having spent around an hour on the phone to my team leader, ETS's head office and their 'helpdesk' it emerged that no-one could give me the simple answers I was looking for.

It seems ETS management are keeping not only the markers in the dark but also the call centre they are operating as a 'helpdesk'. Markers are currently relying on rumour and conjecture for information, much of which propagated by the TES forums. If ETS hadn't shut themselves away they would realise this and attempt to quell these rumours by releasing a statement or providing progress updates. They have remained silent. I believe that they are desperately trying to avoid this going public.

The organisation takes no account for teachers' busy schedules (a whole different story) and seems woefully unprepared. An absolute shambles, it does not bode well for the rest of the marking process. 'Teething problems' do not justify the ineptitude in the management of this organisation.
David Sharples

E-mail this to a friend

Printable version