Surgical Noting

This concept prototype allows surgeons to take medical notes with an Apple pencil on an iPad.

My Role

UI & UX Design
Front-end Development

Time Frame

2 -3 Weeks
Early 2019

Tools

Pen & Paper
Illustrator
HTML
CSS

Team

Gabriel Freire - Mobile Engineer
Richard Garrison - Clinical Safety Officer

Brief

The Noting module or “Surgeons Notes” is a concept prototype designed to be part of the Theatre Checklist of procedures. It allows the Surgeon or medical staff to write notes with a pen or stylus on a touch device, to reduce the reliance on paper-based workflows.

There was a period of down time where nothing was urgently needed, so I was tasked with looking at surgeons note taking. I was given free-reign to experiment, my role was to lead the UX/UI Design and Front-end Development. I was responsible for all designing, prototyping, ideation, testing and the majority of development.

Business Goals

Increase revenue by spending resources on research and development which can lead to new products and features.

User Goals

Easily annotate a body map on a mobile device.

Project Goals

Create a mobile version of the body map as at part of the theatres safety checklist.

Scope and restraints:

ISB 1500-1508: Common User Interface (CUI) guidelines which set information standards for use in health ITsystems
Majority of the Product team work in another country.
Users in another country.

User Demographics

The user demographics for the app is extremely broad as the users can vary from 20 to 65 years of age, they also have varying technical skill levels.

Research

When a patient requires surgery they must go through a series of checklist procedures which are recorded with multiple paper forms. One of these forms is a body map image that the surgeons or medics can draw on and add notes about the operation.

I wanted replicate the intuitive action of hand writing on mobile and experimented with the capabilities of the iPad Pro and Apple Pencil.

I researched different drawing apps such as Adobe Illustrator Draw and Adobe Photoshop Sketch. I really enjoyed using them in conjugation with the Apple Pencil and decided to try something similar. I also thought it would be useful to drag and drop a point of interest or inputted text on the body map and looked at different libraries such as Draggabilly and Interact.js.

Picture Ipad Pro and Apple Pencil — Ipad Pro and Apple Pencil
Source Apple.

Scan of a body map from a physical NHS form

Process

I sketched out different ideas and experimented with the option of having both the front and back body maps displayed simultaneously or having one displayed at a time, with the option to flip from front to back. After consultation with our clinical safety officer Richard, we decided that having front and back visible at both times would be the best way to reduce clinical errors.

For the user interface tools, I wanted to keep it simple and have as little visual clutter as possible. It had a pen tool to allow freehand note-taking or drawing, a drop tool, to allow the user to drag a transparent SVG shape to a point of interest on the body map and a text tool to allow to allow written annotation.

>Wireframe sketch noting UI — Wireframe sketch of noting UI
This would have the option to flip the body map between the front and back.

> Second Wireframe sketch noting UI — Wireframe sketch noting UI
The option to display front and back simultaneously was the best way forward.

>Third Wireframe sketch noting UI — Wireframe sketch noting UI
I put the menu in the centre between the body maps.

>Fourth Wireframe sketch noting UI — Wireframe sketch noting UI

I decided the best way to approach the design was to build an interactive prototype. I wanted to have a responsive design that scaled to any device and iterate it quickly as we tested it. This would be impossible to recreate with prototyping software. The best tools for the project were CSS, HTML Javascript and SVG.

I recruited Gabriel to help me with Javascript when necessary. He had rewritten the noting module for MAXIMS enterprise and changed the output from XML to SVG, so I figured he could add that library to our prototype with minimal development. With him onboard I could develop the UI and he could add the SVG freehand drawing and drag and drop functionality.

After two days the first prototype was finished. For simplicity I made a HTML page and set up a Bsersync Gulp task to cast it to the iPad Pro, so I could use the apple pen. I tested and modified the prototype on the fly from there.

Screenshot of the Noting UI
This was the first version we built.

The drag and drop worked smoothly, we could easily position the SVG circle to any part of the body and annotate it with text from the numpad that could also be repositioned with the apple pen. With a little tweaking, Gabriel lined up the X and Y coordinates to correspond with the tip of the apple pen and we were able to write with the freehand drawing tool. After testing it out we added a refresh option to the menu that would allow you to erase the previous notes and start again.

Second screenshot of the Noting UI — Screenshot of the Noting UI
I added a refresh button to the menu.

We then proceeded to test the prototype with the Head Surgeon and Chief Nursing Information Officer from Musgrove Park hospital, as they were over for meetings in the Irish office. They enjoyed using it and had some feedback on how to improve it.

The surgeon thought it would be useful in a surgery theatre and he could easily write notes on the body map if he was wearing his surgeons scrubs and gloves. The Nurse Information officer thought we could replace the drag and drop options to the bedsore grading system that was on the form original form. She also suggested an option to choose between different colours for the freehand pen tool.

Upon further testing I could see we that we also needed an undo button, this would allow the correction of singular actions, as the refresh button wiped all the notes and the user would have to add all the notes again.

>Wireframe sketch of menu options — Wireframe sketch of menu options

Screenshot of the first version of the colour picker
The pen icon would change colour based on the chosen colour.

Screenshot of the second version of the colour picker

Screenshot of the third version of the colour picker
I separated the colour option from the pen tool to create a new colour menu item.

Screenshot of the fourth version of the colour picker

Screenshot of the fifth version of the colour picker
In the final version the submenu was rounded to differentiate itself from the grading submenu.

We took all the feedback and improved the menu options on the prototype. I added an expandable menu that displayed different bedsore grades and a new menu item that would allow colour selection for the Pen Tool.

Screenshot of the first version of the draggable grading options submenu
In the first version the submenu only had minimal options.

Screenshot of the second version of the draggable grading options submenu
The second version implemented the grading system.

Screenshot of the third version of the draggable grading options submenu
The final version.

Final Concept

Image of the concept working on Ipad — Photo of the prototype working on iPad
This was the final version.

Video demonstrating adding text, freehand drawing, and bedsores on the surgeons noting prototype

Reflections

It was interesting to experiment with new technology using the Apple Pen and iPad pro. Pen and paper is close to my own heart, so it was fun to create a similar experience on mobile. While these ideas still in the exploritiry phase, I can see how this natural method of note taking on a digital device will be of benefit to hospitals in the future.

←

Digital Whiteboard

A brand new product designed to replace traditional analogue whiteboards which allows real time Admission, Discharge & Transfer of patients on mobile.

Go to Case Study

→

Electronic Observations

A medical app designed for clinicians to monitor vital signs and calculate national early warning scores at the patients bedside.

Go to Case Study