Friday, 27 May 2011

Rubbish In - Rubbish Out

A couple of comments on here recently by a magistrate made me realise that it might be instructive if I explain how court Pre Sentence Report's are written nowadays, or should I more correctly say 'generated'. Because of course they are all produced from OASys, but this has never been explained either to judges or magistrates. Just for clarification, I am talking about what are now termed Standard Delivery Reports for either the Crown or Magistrates Court.

Many years ago these were called Social Enquiry Reports which I think pretty well described what they were about. Following a guilty plea or finding of guilt, the judge or magistrates requested a report from a Probation Officer before passing sentence. They wanted to know something about the person, their home situation, any problems, why the offence was committed, what the likelihood of repetition might be and finally a suggestion as to a possible sentence.

An absolutely core part of the job requiring a high level of skill, insight, perception and communication ability in being able to convey an often complex story in a cogent fashion and all in about three pages of A4. I have to say one of the best parts of the job in my view. But before I leap straight in, I need to explain how they used to be written and before the introduction of OASys, the Offender Assessment System, first in paper format and then in electronic form, the so called E-OASys. 

All reports start in the same way, namely at least one interview. This is normally conducted in the office, or in an interview room at the local remand prison if the defendant is remanded in custody. In the case of the former, normally three weeks are allowed and in the case of the latter, normally two weeks. An additional home visit used to be the norm, but this has generally fallen by the wayside. An essential pre-requisite is a full copy of the prosecution disclosure, including statements and a complete print out of previous convictions from the Police National Computer. In complex cases involving any areas of concern, it would be my practice to conduct additional interviews and contact other agencies with knowledge of the defendant. This not only serves to test the person's motivation, it also adds to the report being more authoritative.

In the very early days, reports were handwritten or possibly dictated to a shorthand typist in the luxury of your own office, but for many years my practice was to use a small portable dictating machine, typically whilst pacing around a quiet interview room. Once transcribed, it would be corrected and tweaked, ready for signing and sending off to court. There were a number of forms to complete, which increased over the years of course, but essentially that was it. A skilled officer, in an emergency, could complete the whole task from beginning to end in about two hours. Many a time I have been dictating either long into the night at home, or very early in the morning before dropping the tapes off at work. Productivity was high in my view and each report was prepared with pride in a job well done.

Things started to go wrong when the paper version of OASys came along. A massive document amounting to some forty pages, it had to be filled in before you started writing the PSR. There were seemingly never ending questions to be answered about education, employment, drug use, alcohol, relationships, accommodation, offending, mental health etc etc etc, but all linked to possible risks the person might pose. To be absolutely frank, when introduced I could not believe that no thought appeared to have been given as to how long it would take to fill in. I once asked the Chief at an open forum if she had any idea how long it took to fill in and she admitted she didn't. "How long does it take Jim?" she asked. "About three hours" I replied to gasps of "and the rest!" So at one stroke the time to write a PSR more than doubled and our productivity halved. Not surprisingly, for busy probation officers, the temptation was to write the PSR first and sometimes forget to do the damned OASys. Disciplinary action followed of course. 

But even this option disappeared when E-OASys was introduced. Now you have to be tied to the damned computer in order to fill the electronic version in and it does not let you start on the PSR until it has been completed fully. In my experience this takes between two and three hours even for an experienced officer and in the process numbs the brain completely. There's no longer any option of working at home. You have to complete the process in a noisy open-plan office, in between seeing clients. As a consequence, there is precious little intellectual energy left in order to move on to the PSR itself. But this is the really clever bit - you push a button labelled 'create report.'  

I believe there is a saying in computing that goes something like 'rubbish in - rubbish out.' If you are a sentencer, ponder on that statement next time you are reading a full SDR. As you try to make sense of it, consider the following:- 'why does it seem so formulaic?' or 'that's the fifth time the author has told me they have a drug problem' or 'it does seem repetitative' or 'it doesn't seem to say anything constructive' or 'surely there's a word missing here?' or 'that doesn't make sense' or 'I'm sure that contradicts what it said earlier on.' You see, no matter how careful you've been in completing the OASys in full grammatically correct sentences and not text messages, the computer still has difficulty welding all the information together into a coherent, intelligible whole. But then that shouldn't be too surprising should it? After all 'writing' is a human activity requiring a degree of intelligence.

If you are reading an SDR and any of the above appears familiar, then what you are reading is the result of a modern day computer-generated OASys PSR, complete with a silly little graph or chart. All reports have to be completed in this way, but it's something probation management have been loathe to explain to sentencers for some reason. It might possibly be because of another little annoying aspect of the damned system - the inability to make any corrections because OASys locks the document once completed. I cannot tell you how frustrating it is to be a Court Duty Officer, realising there are major errors in a report, but powerless to correct them because it's locked. Pre-OASys, last minute alterations were easily made at court on a remote computer terminal - but not any more, so reports are regularly submitted riddled with wrong or misleading information in addition to missing or mispelled words.

Now before anyone is tempted to say 'oh our reports aren't like that'  I will explain why that is. Officers like myself, who might have been around for some time and believe in doing a good professional job for the Courts and the benefit of our clients, at the point when the report has been computer-generated, throw it away and start again. 

Can you see why so many of us are stressed and depressed? The work load has doubled for conscientious officers who take pride in the job. No one likes filling in pointless bureaucratic forms for no purpose, and OASys does not even deliver the sound assessment of risk it was designed to do. As a method of writing PSR's, it beggars belief that people of sound mind thought it was a good idea. How can a computer be expected to write a report that makes sense, but more importantly make a cogent and convincing argument for a particular course of action? Well the answer is simple. It can't of course. Management have slowly begun to realise this and hence the now widespread use of vastly inferior Fast Delivery Reports. You see the real benefit of FDR's is that they don't require OASys in order to be completed. 

   

14 comments:

  1. The world is full of bad systems written by programmers for programmers. For over 30 years "management consultancy" firms have chanted their mantra ("It's all about process") as a pre-cursor to getting a software development contract. It has been fundamentally important to their success. Every time one has to deal with a call centre, a utility company or member of the 'financial services' industry, one endures another example. We are now simply extensions to computer programs.

    I have no doubt that somewhere in the country a "management consultant" (we used to call them software salesmen) is working on a proposal to enhance the system. If it was me, I'd be suggesting that the perp should perform their own assessment. By front ending it with some questionnaires, hooking it up to polygraph systems etc. & putting a box with some flashing lights on the top should be a nice little earner.

    Blair's great legacy has been the deprofessionalisation, de-humanisation and dumbing down of many fields. His heir seems to have picked up the baton.

    Ray.

    ReplyDelete
  2. Thanks for this insight, which doesn't surprise me one bit. I've sent the link to my whole bench.

    ReplyDelete
  3. Very informative. I think is says a lot about the marketing departments of the software industry who design these systems.

    ReplyDelete
  4. A probation officer27 May 2011 at 20:39

    Your experience is not representative of all probation areas Jim. I am consistently impressed by the quality of reports presented to the courts in my area. (even if they are underpinned by Oasys) -This is down to the skills and professionalism of the staff completing them (must admit it does sometimes take a lot of tinkering about with - to achieve court standard quality). OASys has standardised reports (which can be sometimes seen as dull and formulaic) that I will concur - however there are benefits as well since this can contribute to a more comprehensive and thorough report than would have been produced. Constantly remind myself that OASys is a tool and don't think it helps to see as an unnecessary hinderance. That said I will be greatly relieved when it is modified(shortened) and works/assists probation staff (as a tool should do). I really enjoy following your blog - like minded on most matters! keep up the good work have been spreading the word amongst my colleagues about this blog! From a software perspective - The Sentence Plan - now that Is a nightmare! - whoever designed that should hang their head in shame!!!

    ReplyDelete
  5. Ah the Sentence Plan! That's another bit of OASys I've not tackled yet!

    Thanks for commenting,

    Jim

    ReplyDelete
  6. The way in which my area use on-the-day "oral" delivery reports means we get to use our skills as report authors without the hassle of a rigid, formulaic approach (like FDRs and OASys) - they're starting to very much resemble old social enquiry reports.
    On the complex ones (that is those beyond "yes, suitable for unpaid work/curfew") we no longer deliver them orally - preparing a written document on the template to hand up - often 2-3 pages worth of free text of offence analysis, social needs & risk assessment and a relevant proposal. We frequently get positive feedback from the benches.

    I am not in the slightest bit envious of the PSR authors who are confined to the restraints of an OASys generated report!

    Coincidentally, I came across an old Social Enquiry report this week. On a life case dating back older than I am. It was two A4 pages and I was gob-smacked at how little useful information it really contained; especially relevant to the horrendous crime it was prepared on. The end conclusion was that this man held a deep-rooted anger towards a member of his family - that had nothing to do with the offence. I was baffled.
    So it is worth noting, that without doubt there were excellent SER's "back in the day" but there were also less than satisfactory ones. Much like now with PSRs. Rose-tinted spectacles.....!


    (also, I've been reading all your entries - good to see so many. Restrictions on the work systems prevent me commenting)

    ReplyDelete
  7. A177,

    Good to see you back and I understand the difficulty in not always being able to comment.

    That's very interesting to hear about a return to 'oral' reports, even though a pro-forma is being used now. All to avoid OASys of course - it's crazy!

    I appreciate that some SER's were not that good, but there was a halcyon period from about 1990 till OASys came along when PSR's done in free-style were pretty good I think. Always the odd exception though.

    Thanks for still reading, it's good to know there's an audience out there.

    Cheers,

    Jim

    ReplyDelete
  8. jim

    I think oasys is ok. Some of the points you make above are practice issues not system issues. You can edit reports prior to submitting them (in the word doc). You should not be submitting reports with missing words and spell check is available (in the word doc again). I just write my report in the text boxes and it pulls thru to the word doc. Simple. I write my report not the computer. Rubbish doesn't come out because I don't put rubbish in. There are comments that I include in the oasys which aren't meant for the report and I simply delete them.

    You say the following "Now before anyone is tempted to say 'oh our reports aren't like that' I will explain why that is" but I don't think you actually have explained "why that is".

    Oasys is just a tool. It's not perfect of course. There are too many rules about it's use (timeframes, reviews etc). But it's still just a tool. We just just need to make it work for us.


    Keep it up please. I enjoy the blog and normally agree with you but not today!

    ReplyDelete
  9. A wrench is a tool. A gun is a tool. A sewing machine is a tool. Just because you have a tool at your disposal doesn't mean it's the right one for the job. Some might say a brain is a tool - in my experience it is one that is rarely used where OASys is concerned.

    ReplyDelete
  10. I need to respond to the person who thinks "OASys is ok".

    "I just write my report in the text boxes and it just pulls thru to the word doc. Simple"

    With the greatest respect you are not writing a report, you are answering questions put to you by OASys - the computer software joins all your disjointed text boxes together in order to form the report. That's the theory at any rate.

    Are you seriously telling me that the result only needs a bit of tweaking and editing in order to get it into a professional document ready for court where someone's whole future can be at stake? I'm prepared to wager that it is not because I've read plenty of such examples over recent years and they would fail any creative writing test.

    In my experience in order to satisfy the high standards of professionalism that the Courts and our clients are due the resulting mangled word doc has to be discarded and completely re-written. I think this is largely a generational issue. Just ask any long-serving colleague what they do when writing a report through OASys and I guarantee they virtually re-write it.

    The system is not fit for purpose and it's simply a barmy way to try and write a court report.

    Thanks for commenting - I do appreciate it, but there are very serious concerns about OASys in lots of quarters and if it's not addressed the whole professional basis of the Service is in question.

    Cheers,

    Jim

    ReplyDelete
  11. Hi Jim,

    I agree with your points about the formulaic and prescriptive nature of OASys and how this can hugely impact on the quality of reports. I too feel that the system could be much improved. I remember reading an article during my training, in which it was argued that OASys far too readily reduces the individual to little more than the sum of sections 1-13. This is an idea that has really stuck with me, and one I am mindful of when proofing my reports.

    However, the general tone of this entry makes me rather uncomfortable. The implication is that anyone who does not re-write their reports entirely, once 'generated', is doing a bad job and can't possibly submit a decent piece of work. Are you saying then, that all officers who have qualified in the past decade or so (and whom will have learnt to write their reports in this way) are doing a poor job?

    The suggestion makes me feel a little indignant. Yes, I think we would benefit from a partial return to SIR style assessment. Yes, I think that OASys allows for incoherent and illogical reports – if the author doesn’t take the time to thoroughly proof-read and revise them. What I don’t agree with is the insinuation that, in making use of OASys to structure my PSR, I will only ever achieve a shoddy end product. Clearly some staff will have a greater capacity for report writing, larger vocabularies and more distinctive style; a poor report cannot be blamed on OASys alone. As for failing any creative writing test – when did a degree in language and literature become a prerequisite for the job?

    I put a lot of time and effort into ensuring that my reports are as good as they can be, given the obvious time constraints. It’s fairly disheartening to hear that more experienced colleagues have such little regard for the work of more recently qualified officers.

    Right, whinge over.
    C

    ReplyDelete
  12. Clare,

    Thanks for commenting - I understand entirely your sense of indignation. The points you raise are so important, hopefully with your agreement I'd like to write a post based on them. Should be ready by tomorrow.

    Cheers,

    Jim

    ReplyDelete
  13. Of course. I look forward to reading it.

    ReplyDelete
  14. A Purple Judge21 June 2011 at 12:28

    A very interesting insight. Most informative.

    ReplyDelete