BASELINE CASE STUDY B:

An audit of the confidence levels, usage and experiences within our college community

November 2011 – January 2012

Background overview

 

As part of the initial base lining phase of the JISC Digital Literacy project ‘Wordle’ at Worcester College of Technology we decided to ask our whole community about their technology usage, experiences and the perceived levels of confidence and proficiency.  We felt a rich profile of a wide range of participants would offer a unique insight into our college community as we develop our ideas about where to focus the key levels of OCN support within our college environment.

 

 

Participants Profile

 

This was a college wide survey and was distributed amongst a range of staff and student stakeholders within our college who interact with each other at the chalk face of education or who offer key support structures. In total we had approximately 250 respondents to our survey.

The student participants ranged from school age right through to higher education courses and were taken from a random cross section of students from the following courses:

 

·         School Age learners that come into our college to study a diploma

·         FE Vocational Students – Level 1

·         FE Vocational Students – Level 2

·         FE Vocational Students – Level 3

·         HND & HNC Level Courses

·         A-Level (AS & A2)

 

 

The staff participants were from the three main roles that interact with students in their daily learning or roles that underpin the support needed for study skills and using the library and digital resources for learning.

These were taken from:

 

·         Teachers

·         Study Centre/Library Team

·         Personal Tutors

 

 

Methods

 

This took the form of a survey questionnaire that was broken into sections that asked questions about their technology experiences, preferences, usage and proficiency confidences in their skills. This offered us a very detailed picture of their individual profiles and attributes. It was chosen that the surveys be distributed in paper form rather than an online survey to ensure that all users could participate and not just those familiar or comfortable using online surveys. We felt that as this project was to conclude with the development of OCN modules to underpin any weaker skills we needed to get a richer picture of all digital literacy skills and usage levels both the strong users and the very weak.

 

Conducting the survey in this manner did bring the consequence of having to enter the data manually into Microsoft Excel which was time consuming. However this also meant we were able to bespoke design our data entry leaving us with the opportunity to further interrogate the results as we look into the granular detail of skills and confidence levels for the OCN unit development later in the project. This potential is not always available in many ‘off the shelf’ survey packages as they generally offer only the more generalised analysis.

 

Our surveys also collected further data such as their learning level, group, age and sex to provide us with the much finer detail should we need it at the deeper analysis stage.

 

Within the survey we also asked the question ‘I would be happy for learning to be delivered through my social networking and communication tools (e.g. Twitter, FaceBook, Mobile phone)’ to gauge their opinion about this area. We additionally asked them to describe why they answered yes or no to this question and their reasoning behind this. This is something we as a college have always been interested to investigate, primarily to see if this offers a potential means of supporting their learning or digital skills development beyond the classroom.

 

Results

 

At the baseline level we were very keen to find out the basic differences between staff and students to see if there were any similarities or differences between the types of tools they used and those they felt most comfortable and confident using. Also we were very interested to see which tools were dominant in their usage both educationally and socially as college staff and students.

 

Much later in the project the data will be interrogated much further to investigate any key themes and how they compare to the other perspectives we have explored, such whether their interpretations in Case Study A or perceptions of each others skills in Case Study B correlate with confidence levels from this study.  The confidence levels will also be validated by an information literacy task to see if the skills confidences match the reality or whether any of our participants have over or underestimated their skills and proficiency.

 

Any emerging themes will be taken into consideration to inform the design of the OCN units to help us target where help is needed most and where certain skills can be harnessed and nurtured.

 

At this baseline stage we have looked at those figures that have exposed the most significant key differences between the students and staff as a direct comparison. This is not yet broken down into the more granular detail of the ages and stages of their learning or staffing roles.

 

The comparable results of staff and students were very interesting in the following questions:

 

Ownership and Experience of Use

 

 

These first three graphs show a selection of questions asked about the more academic tools and as it indicates the teachers’ usage or experience of academic tools is significantly higher. However when you look at the more social uses of technology you get more of an opposite response where learner usage of technology for social use is more pronounced.

 

 

 

 

Confidence Levels: Information Skills

 

Beyond their access habits or experiences we then asked about how confident they were with their academic tools and processes. Interestingly the figures in the tables below are pretty similar however the staff do have slightly more confidence in this area which is indicated by the larger percentage of not used/can’t use figures in the student table (as seen in purple). This is possibly due to the teachers’ longevity of their experiences and using such tools on a daily basis in their professional context. FE learners and school age students may also not yet have been exposed to the more academic tools often associated with the much higher HE and undergraduate study programmes

 

 

Confidence Levels: Social Tools

 

Beyond their access habits or experiences we then asked about how confident they were using their social tools. There is predominantly a difference here between students and staff. An interesting 31% of staff claimed they have not used/can’t use Facebook whereas 81% of students do over the staff 44%.

Interestingly the figures for online collaboration are very similar so they do seem to have a similar profile in terms of use or confidence yet may apply themselves differently through different tools, which clearly in students appears to be Facebook.

 

 

Confidence Levels: VLE Tools

 

Next we asked about their confidence levels when finding information or uploading information on the VLE. I think it goes without saying that students confidence maybe less in the uploading content section due to them not uploading to content to build a VLE course, however their interpretation could be that they have been asked to upload content onto an activity or for an assignment.

For staff they are more confident than students but overall those who answered highly confident/confident in the staff table for uploading only amount to 41% which is worryingly low considering the college has moved over to a blended learning approach in their FE courses.

Using the VLE to find and locate resources was better but on this occasion the staff were slightly less confident which again is concerning for the blended learning agenda at the college.

 

 

Confidence Levels: General Tools

 

With the more generic tools that are used regularly such as Microsoft Office or the day to day functions or issues you would encounter using technology the picture is again very similar when it comes to confidence levels and not a wide gap as many commentators have suggested. There are some areas where student feel they are more confident and that is when using PowerPoint or attending to technical matters such as installing software on a PC, sorting our computer viruses and using both MAC and PC platforms. However for both parties these figures are still pretty low indicating on both sides there is room for more support in this area.

 

 

 

 

Using Social Media for Education

 

Interestingly we also asked the question:

I would be happy for learning to be delivered through my social networking and communication tools (e.g. Twitter, FaceBook, Mobile Phone)

 

 

This was a very balanced result with a very similar profile of agreement between staff and students. Overwhelmingly with a near 70% agreement from both parties that they would not want to

 

We also asked them to:

Explain their reasons why they answered YES/NO to this question.

 

The reasons given for the YES/NO answers are detailed in a seperate file 

 Generally the comments suggested that overall both learners and staff social and academic tools as very different with a distinct boundary that they have different purposes.

 

A very interesting finding was the mapping of confidences. Although this was subjective in terms of participants judging themselves we decided to map the confidence levels of students and staff to compare their overall confidences in the four main skills areas.

 

 

As you can see from the diagram above the students appear to have more confidence in the use of social and multimedia tools, whereas the staff have more confidence with their information literacy skills. What was interesting is the 'ICT General' skills where staff and students appear to be very close in their perception of confidences. These however are the more day to day baseline skills (mainly office applications, emails, printing and storing information electronically) that have been embedded in practice for well over a decade. This mapping does reveal that learners tend to be more confident socially whereas staff focus mainly their attention on academic practices. This could indicate their usage patterns for example young people versus those with more professional lives.

 

 

Conclusion

 

It needs to be noted that the survey holds extensive data and there are many more figures to be examined from this survey. These will be examine in due course in greater depth throughout the project as we examine areas that need the most support for writing the OCN modules.

 

This was a very interesting exercise because the results were not necessarily matched to broad assumptions made about the differences between staff and students in terms of the confidence levels of some tools.

 

To be expected the was a difference in the experience of the more academic tools, such as teachers had more experience and confidence in these than students, and the reverse was true of social tools which students dominated in their experiences in these.

 

What was most interesting though was the confidence levels in the use of everyday tools such as Microsoft Word and PowerPoint there were differences but not anywhere near as significant as were are always led to believe in the hype of the ‘digital natives’ and ‘digital immigrants’ classification debate. The use of a VLE to locate resources was also pretty matched between staff and students.

 

These figures will be examined in more depth at each age and stage of education to plan and develop the right level of support for the OCN qualifications.