Evaluating my writing groups

This is a long post so I have included headings to allow you to navigate it more easily:
Introduction
Why evaluate?
What forms have I used?
Findings
Conclusions
References

Introduction

I lead three regular therapeutic writing group sessions for inpatients in mental health units in Cumbria, including a weekly group in a psychiatric intensive care unit (PICU). The people attending these groups include regulars who have been coming every week for months and new people I haven’t met before.

Hadrian Poster

Then there is my monthly group, which I started so that service users would have a writing group to go to after they leave hospital. This group is open to anyone who is interested in using writing to help their health and wellbeing.

Depending on the individuals present I use a wide variety of writing exercises, including mindful writing, which is good for calming and focussing the mind and especially helpful in the PICU, and flexible writing activities, with the aim of inspiring group members to write whatever they want or need to write, for example, immediate thoughts, memories, stories, poems. You can find a lot more detail about the writing exercises I use elsewhere on this blog.

Why Evaluate?

The reasons for evaluating my groups that occurred to me when I was planning this post were:

Effectiveness – the main reason to evaluate is to make sure the groups are effective, which for me means they need to be enjoyable and therapeutic – so those are the aspects I have tried to measure with the forms I have used so far.

Learning – another reason I evaluate the sessions is to improve my practice. No-one knows the ‘right’ approach to therapeutic writing in mental health. I happen to think there is no one ‘right’ approach and that different people with different symptoms and at different stages of recovery need different writing interventions. The comments people write on individual evaluation forms, and the overall anaysis of the ‘tick boxes’ on 100+ forms, all help me improve my practice.

Evidence – therapeutic writing is not (as far as I know) commonly practised in the NHS here in the UK. In the USA I believe it is more widely used. So, we need to evidence the benefits of therapeutic writing sessions to ensure they continue and increase.

What forms have I used?

My first group in an adult mental health ward was a pilot. My view was that the group would be a ‘therapeutic’ writing group. However, I could find no evidence of the therapeutic benefit of writing groups in mental health units. How could I call the group ‘therapeutic’ if I had no evidence? I looked for a tool that measures the therapeutic benefit of sessions – not writing sessions, any sessions. What I found was a tool to measure the therapeutic benefit of psychotherapy sessions, the Session Evaluation Questionnaire (SEQ) developed by Stiles, Gordon and Lani (2002). The advantages of the SEQ are that it is not easy for the person completing it to guess the ‘desired’ answers and possibly bias the results; and that it incorporates clinically relevant measures: positivity, psychological arousal, session depth, session smoothness. The main disadvantage is its complexity: it involves the person completing a complicated questionnaire before and after each session – not easy for someone who is acutely unwell. After discussing the SEQ with senior clinicians in the ward, I removed a couple of questions to make it easier to complete. Of course this means I did not use the validated SEQ form, but I believe the results are nontheless true.

The pilot writing sessions were popular, and the evaluations encouraging, so the hospital invited me to continue the sessions in the original ward, and later to start groups in the PICU and another mental health ward. I decided to develop a new, simpler, form that would evaluate whether the sessions were enjoyable, and had any effect on mood, anxiety, and so on. I have now been using this form for over two years, with all my writing groups. I have been happy with the form, but I do plan to improve it over the next month or two Evaluation form 2011 v1

In my work in mental health I work closely with the ward Occupational Therapists (OTs). Laura White, the OT of two of the wards decided to evaluate all the different groups running in one ward for the whole of May 2013. Laura devised a new form with an OT focus: OT Activity Questionnaire. Although the focus of the form is on OT, rather than therapeutic writing, and so (in my opinion) missing a few questions, there are some things I really like about Laura’s form that I plan to adapt and incorporate when I improve my own form.

Findings

1. Evaluation of the pilot sessions (April – August 2010)

This is a summary of the findings from the SEQ evaluation of the pilot writing group:
24 pairs of before/after SEQ were completed
Participant satisfaction averaged 5.7 out of 7.0 (81% satisfied)
Positivity increased by session for 75%
Psychological arousal increased in 46%, decreased in 21%, and was unchanged in 33% (whether this is good or bad is difficult to judge because it depends what the level of psychological arousal starts at as to whether it needs to go up or down)
Session depth averaged 5.1 out of 7.0 (73%)
Session smoothness averaged 5.0 out of 7.0 (71%)

The evaluation of the pilot group was written up in summary form in the Cumbria Partnership Journal of Research Practice and Learning (Ross, 2012, see section on ‘Inpatient writing sessions’).

2. Evaluation of inpatient mental health groups (April 2011 – April 2013)

Looking only at my inpatient groups, and based on my current evaluation form (178 forms):
88% of inpatients definitely enjoyed the group and 99% at least partly enjoyed it
The group helped people with…
64% – lifting mood
63% – expressing themselves
61% – relaxation
52% – anxiety
52% – talking to others
41% – understanding themselves.

Inpatient Chart

One of the questions asks what was the best thing about the group and the most frequent answers are represented by the Wordle below. Some people stated benefits that are not currently covered by the evaluation questionnaire: increased confidence, a sense of achievement, remembering happy memories, creativity, a new interest (not always in writing), sharing thoughts with others, listening skills, using my imagination, moving ahead with my thinking, trying to overcome barriers, my old self came back, realising I can do something good, remembering how lucky I am, getting everything out of my system, learning about ‘flow’, inspiration to start a journal again, thought, looking from a different perspective, relief from boredom, inspiration, something different, an idea for something to do in the future, activated my mind, getting my mind working on different things, thinking in depth about things, that it’s ok to put your feelings on paper, communication skills.

Wordle: Evaluation of writing groups

3. The OT Activity Evaluation (May 2013)

The OT activity questionnaire was only completed by 13 inpatients, 100% of whom stated they enjoyed the group

Benefits stated:
77% – social contact
62% – structured time
46% – distraction
38% – change of environment
0% – increased physical activity (not a surprising result this!)

54% of people said their interest in creative writing was increased by attending the session.

69% said the group made them feel better – 33% said much better and 38% said a little better.

There are obvious questions missing from this evaluation from a writing practitioner’s point of view, for example, one person wrote on the form that the main benefit for them was increased mental activity. Even so, the results are very good, especially bearing in mind that the people attending the sessions are mentally very unwell – anything that makes 69% of people feel even a little better is good as far I am concerned!

Conclusions

A large majority of attendees enjoy the sessions and the findings from the pilot suggest that the majority of people experience the sessions as deep and smooth. The main perceived benefits seem to be: increased positivity/lifted mood, help with self expression, relaxation, social contact/talking to others and decreased anxiety.

Comments written on the forms suggest that the main evaluation form I have been using covers only a few of the benefits that inpatients feel they get from the writing sessions. Some of the benefits stated in the comments could be incorporated into the re-design of the form.

References

Ross, C. A. (2012). Evaluation of Year of Writing workshops. Cumbria Partnership Journal of Research, Practice and Learning 1(1). Retrieved 30 June 2013 from http://www.cumbriapartnership.nhs.uk/uploads/Journal/CPJRPL%201%201%20Spring%202011%20p17%20Evaluation.pdf

Stiles, W. B., Gordon, L. E., & Lani, J. A. (2002). Session evaluation and the Session Evaluation Questionnaire. In G. S. Tryon (Ed.), Counseling based on process research: Applying what we know (pp. 325‐343). Boston, MA: Allyn & Bacon. Edited version retrieved 30 June 2013 from http://www.users.miamioh.edu/stileswb/session_evaluation_questionnaire.htm

Advertisements

About Carol Ross

Interested in therapeutic writing.
This entry was posted in reflective practice, Research and articles and tagged , . Bookmark the permalink.

2 Responses to Evaluating my writing groups

  1. ChannaChanna says:

    Thanks Carol for posting this. I hope it will set off a good discussion about evaluations. I am making a start on a post on my blog to share my experiences and tools. It is not out there yet, but will soon be, with a referral to your blog as the inspiration.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s